Abstract
Introduction Checkpoint inhibitor treatment has proven successful for advanced melanoma. However, a significant fraction of patients does not experience benefit from this treatment, that is also associated with potentially severe toxicity and high costs. Previous research has not yet resulted in adequate biomarkers that can predict treatment outcomes. The present work is the first to investigate the value of deep learning on computed tomography (CT) imaging of melanoma lesions for predicting checkpoint inhibitor treatment outcomes in advanced melanoma.
Methods Adult patients that were treated with first line anti-PD1 ± anti-CTLA4 therapy for unresectable stage IIIC or stage IV melanoma were retrospectively identified from ten participating centers. Up to five representative lesions were segmented volumetrically on baseline CT; a deep learning model (DLM) was trained on the corresponding volumes to predict clinical benefit, defined as stable disease for a minimum of six months, or response at any time during follow-up. Optimal hyperparameters and model types (Densenet, Efficientnet, Squeeze-Excitation ResNet, ResNeXt) were iteratively explored. The DLM was compared to a model of previously identified clinical predictors (presence of liver and brain metastasis, level of lactate dehydrogenase, performance status and number of affected organs), and a combination model consisting of both clinical predictors and the DLM.
Results A total of 730 eligible patients with 2722 lesions were included. Rate of clinical benefit was 59.6%. The selected deep learning model was a Squeeze-Excitation ResNet with random initialization, trained with the Adam optimizer. The DLM reached an area under the receiver operating characteristic (AUROC) of 0.607 [95% CI 0.565 – 0.648]. In comparison, a model of clinical predictors reached an AUROC of 0.635 [95% CI 0.592 – 0.678]. The combination model reached an AUROC of 0.635 [95% CI 0.595 – 0.676]. None of the differences in AUROC were statistically significant. The output of the DLM was significantly correlated with four of the five input variables of the clinical model.
Discussion Although the DLM reached a statistically significant discriminative value, it was unable to improve over previously identified clinical predictors. The most likely cause is that the DLM learns to detect a lesion’s size and organ location, which is information that is already present in the clinical model. Given the substantial sample size and extensive hyperparameter optimization, this indicates that the predictive value of CT imaging of lesions for checkpoint inhibitor response in melanoma is likely limited. The present work shows that the assessment over known clinical predictors is an essential step for imaging-based prediction and brings important nuance to the almost exclusively positive findings in this field.
Competing Interest Statement
AvdE has advisory relationships with Bristol-Myers Squibb, MSD Oncology, Amgen, Roche, Novartis, Sanofi, Pfizer, Ipsen, Merck, Pierre Fabre and has received research study grants not related to this paper from Sanofi, Bristol-Myers Squibb, TEVA, Idera and has received travel expenses MSD Oncology, Roche, Pfizer, Sanofi, Pierre Fabre and has received speaker honoraria from BMS and Novartis. JdG has consultancy/advisory relationships with Bristol Myers Squibb, Pierre Fabre, Servier, MSD, Novartis. PJ has a research collaboration with Philips Healthcare and Vifor Pharma. MBS has consultancy/advisory relationships with Pierre Fabre, MSD and Novartis, none related to current work and paid to institute. EK has consultancy/advisory relationships with Bristol Myers Squibb, Novartis, Merck, Pierre Fabre, Lilly, Bayer, EISAI and Ipsen paid to the institute, and received research grants not related to this paper from Bristol Myers Squibb, Delcath, Novartis and Pierre Fabre. PD has consultancy/advisory relationships with Paige, Pantarei and Samantree paid to the institution and research grants from Pfizer, none related to current work and paid to institute. KS has advisory relationships with Bristol Myers Squibb, Novartis, MSD, Pierre Fabre, AbbVie, Sairopsa and received honoraria from Novartis and MSD and research funding from Bristol Myers Squibb, TigaTx and Philips. TL has received research funding from Philips. GH consultancy/advisory relationships with Amgen, Bristol-Myers Squibb, Roche, MSD, Pfizer, Novartis, Sanofi, Pierre Fabre and has received research grants from Bristol-Myers Squibb, Seerave. All payments to the Institution. HW received honoraria from Merck, Astellas, Roche and travel expenses from Ipsen and Astellas All remaining authors have declared no conflicts of interest.
Funding Statement
This research was funded by The Netherlands Organization for Health Research and Development (ZonMW, project number 848101007) and Philips Healthcare.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
After review by the Medical Research Ethics Committee (NedMec, Utrecht, The Netherlands), this study was deemed not subject to the Medical Research Involving Human Subjects Act in accordance with Dutch regulations. Informed consent was waived.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
Due to confidentiality agreements, clinical and imaging data cannot be made available.
