Recent advances in tissue imaging technologies have led to the generation of massive datasets of spatial profiles of human tissues, taken at micron scale resolution, spanning hundreds of patients, and across tens to thousands of molecular biomarkers. These high-dimensional imaging data necessitate the development of new, AI-based tools to uncover new biology and to support therapeutic developments against diseases. Currently, a major need in the treatment of solid tumor cancers are strategies that can drive the infiltration of T cells into the tumor. In this study, we developed an optimization strategy combining supervised ML and counterfactual explanations to discover clinically feasible tumor perturbations that drive T cell infiltration, and applied our framework to spatial proteomes of breast cancer and melanoma tissue. Our model predicts that altering the levels of four molecules (CCL4, CXCL12, CXCL13, CCL8) in immune-excluded melanoma tissues can increase T cell infiltration by 10 fold, across an entire cohort of 69 patients. Our work provides a paradigm for machine learning based prediction and design of cancer therapeutics based on classification of immune system activity in spatial proteomics data.