Abstract
Spatial transcriptomics technology remains resource-intensive and unlikely to be routinely adopted for patient care soon. This hinders the development of novel precision medicine solutions and, more importantly, limits the translation of research findings to patient treatment. Here, we present DeepSpot, a deep-set neural network that leverages recent foundation models in pathology and spatial multi-level tissue context to effectively predict spatial transcriptomics from H&E images. DeepSpot substantially improved gene correlations across multiple datasets from patients with metastatic melanoma, kidney, lung, or colon cancers as compared to previous state-of-theart. Further, we validated our approach at the Autoimmune Disease Machine Learning Challenge 2025 organized by the Broad Institute of MIT and Harvard, and greatly outperformed other methods, securing first place. Using DeepSpot, we generated 3 780 TCGA spatial transcriptomics samples (56 million spots) of the melanoma, renal cell cancer, lung adenocarcinoma and lung squamous cell carcinoma cohorts. We anticipate this to be a valuable resource for biological discovery and a benchmark for evaluating spatial transcriptomics models. We hope that DeepSpot and this dataset will stimulate further advancements in computational spatial transcriptomics analysis.
Competing Interest Statement
V.H.K reports being an invited speaker for Sharing Progress in Cancer Care (SPCC) and Indica Labs; advisory board of Takeda; and sponsored research agreements with Roche and IAG, all unrelated to the current study. VHK is a participant in a patent application on the assessment of cancer immunotherapy biomarkers by digital pathology; a patent application on multimodal deep learning for the prediction of recurrence risk in cancer patients, and a patent application on predicting the efficacy of cancer treatment using deep learning. GR is a participant in a patent application on matching cells from different measurement modalities which is not directly related to the current work. Moreover, G.R. is a cofounder of Computomics GmbH, Germany, and one of its shareholders.
Funding Statement
We gratefully acknowledge funding from the Tumor Profiler Initiative and the TumorProfiler Center (to V.H.K., G.R.). The Tumor Profiler study is jointly funded by a public-private partnership involving F. Hoffmann-La Roche Ltd., ETH Zurich, University of Zurich, UniversityHospital Zurich, and University Hospital Basel. We also acknowledge funding of K.N. by SwissNational Science Foundation (SNSF) grants 220127 (to G.R.) and 201656, ETH core funding (toG.R.), UZH core funding (to V.H.K.), funding by the Promedica Foundation grant F-87701-41-01 (toV.H.K.), SNSF Prima grant PR00P3-201656 (to K.S.) and funding from the Swiss Federal Institute Of Technology strategic focus area of personalized health and related technologies project 2021-367(to G.R., V.H.K., S.A.).
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
The ethics committee of the Swiss Association of Research Ethics Committees gave ethical approval for the data from the Tumor Profiler Study used in this work. The Tumor Profiler Study is an approved, observational clinical study (BASEC: 2018-02050, 2018-02052, 2019-01326). Additionally, ethical approval for performing research on anonymized, archival patient material was obtained from the cantonal ethics commission Zurich (BASEC Nr. 2022-01854 and BASEC-Nr. 2024-01428) for the "Kidney and Lung cancer with TLS" data.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Footnotes
Extended model evaluation and new experiments (Section 2.6 and Section 2.7). Supplemental files updated.
4.6 Data availability
The kidney, colon and lung cancer HEST-1K datasets were downloaded using the HEST-1K download pipeline outlined here: https://github.com/mahmoodlab/HEST/blob/main/tutorials/1-Downloading-HEST-1k.ipynb, with data available at Hugging Face https://huggingface.co/datasets/MahmoodLab/hes. The Tumor Profiler spatial transcriptomics dataset was obtained from [5]. The lung and kidney cancer with TLS dataset is available at https://zenodo.org/records/14620362. The human lung reference atlas is available at https://data.humancellatlas.org/hca-bio-networks/lung/atlases/lung-v1-0. The lung cancer dataset was downloaded from "http://www.ebi.ac.uk/biostudies/arrayexpress/studies/E-MTAB-13530". The single-cell lung reference atlas was downloaded from https://data.humancellatlas.org.
TCGA spatial transcriptomics samples are available for download on Hugging Face. Access details can be found at https://github.com/ratschlab/DeepSpot.
