RT Journal Article SR Electronic T1 Blind validation of MSIntuit, an AI-based pre-screening tool for MSI detection from histology slides of colorectal cancer JF medRxiv FD Cold Spring Harbor Laboratory Press SP 2022.11.17.22282460 DO 10.1101/2022.11.17.22282460 A1 Saillard, Charlie A1 Dubois, Rémy A1 Tchita, Oussama A1 Loiseau, Nicolas A1 Garcia, Thierry A1 Adriansen, Aurélie A1 Carpentier, Séverine A1 Reyre, Joelle A1 Enea, Diana A1 Kamoun, Aurélie A1 Rossat, Stéphane A1 Sefta, Meriem A1 Auffret, Michael A1 Guillou, Lionel A1 Fouillet, Arnaud A1 Kather, Jakob Nikolas A1 Svrcek, Magali YR 2022 UL http://medrxiv.org/content/early/2022/11/22/2022.11.17.22282460.abstract AB Objective Mismatch Repair Deficiency (dMMR) / Microsatellite Instability (MSI) is a key biomarker in colorectal cancer (CRC). Universal screening of CRC patients for dMMR/MSI status is now recommended, but contributes to increased workload for pathologists and delayed therapeutic decisions. Deep learning has the potential to ease dMMR/MSI testing in clinical practice, yet no comprehensive validation of a clinically approved tool has been conducted.Design We developed an MSI pre-screening tool, MSIntuit, that uses deep learning to identify MSI status from H&E slides. For training, we used 859 slides from the TCGA database. A blind validation was subsequently performed on an independent dataset of 600 consecutive CRC patients. Each slide was digitised using Phillips-UFS and Ventana-DP200 scanners. Thirty dMMR/MSI slides were used for calibration on each scanner. Prediction was then performed on the remaining 570 patients following an automated quality check step. The inter and intra-scanner reliability was studied to assess MSIntuit’s robustness.Results MSIntuit reached a sensitivity and specificity of 97% (95% CI: 93-100%) / 46% (42-50%) on DP200 and of 95% (90-98%) / 47% (43-51%) on UFS scanner. MSIntuit reached excellent agreement on the two scanners (Cohen’s κ: 0.82) and was repeatable across multiple rescanning of the same slide (Fleiss’ κ: 0.82).Conclusion We performed a successful blind validation of the first clinically approved AI-based tool for MSI detection from H&E slides. MSIntuit reaches sensitivity comparable to gold standard methods (92-95%) while ruling out almost half of the non-MSI population, paving the way for its use in clinical practice.Competing Interest StatementCS, RD, OT, NL, SV, M. Sefta, MA, LG, AF are employees of Owkin Inc. J.N.K. declares consulting services for Owkin, France, for Panakeia Technologies, UK and for DoMore Diagnostics, Norway. M. Svrcek declares consulting services for Owkin, France.Funding StatementThis study was funded by Owkin.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:All experiments were conducted in accordance with the General Data Protection Regulation (GDPR) and the French laws and regulations. Medipath data subjects have generally been informed for the re-use of their samples and data collected during the care for research purposes. Medipath has obtained an approval of the "Ministere de l'Enseignement Superieur, de la Recherche et de l'Innovation (MESRI)" for the storage of samples for research purposes and has nominatively reinformed patients for the reuse of their data for Owkin's experiments.I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).Yes I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesThe discovery TCGA dataset is publicly available at the TCGA data portal (https://portal.gdc.cancer.gov). The external validation PAIP dataset is available at https://paip2020.grand-challenge.org/. Other data produced in the present study are available upon reasonable request to the authors.The discovery TCGA dataset is publicly available at the TCGA data portal (https://portal.gdc.cancer.gov). The external validation PAIP dataset is available at https://paip2020.grand-challenge.org/. Other data produced in the present study are available upon reasonable request to the authors.