PT - JOURNAL ARTICLE AU - V. Venugopal AU - R. Takhar AU - S. Gupta AU - A. Saboo AU - V. Mahajan TI - Clinical Explainability Failure (CEF) & Explainability Failure Ratio (EFR) – changing the way we validate classification algorithms? AID - 10.1101/2020.08.12.20169607 DP - 2020 Jan 01 TA - medRxiv PG - 2020.08.12.20169607 4099 - http://medrxiv.org/content/early/2020/08/14/2020.08.12.20169607.short 4100 - http://medrxiv.org/content/early/2020/08/14/2020.08.12.20169607.full AB - Adoption of Artificial Intelligence (AI) algorithms into the clinical realm will depend on their inherent trustworthiness, which is built not only by robust validation studies but is also deeply linked to the explainability and interpretability of the algorithms. Most validation studies for medical imaging AI report performance of algorithms on study-level labels and lay little emphasis on measuring the accuracy of explanations generated by these algorithms in the form of heat maps or bounding boxes, especially in true positive cases. We propose a new metric – Explainability Failure Ratio (EFR) – derived from Clinical Explainability Failure (CEF) to address this gap in AI evaluation. We define an Explainability Failure as a case where the classification generated by an AI algorithm matches with study-level ground truth but the explanation output generated by the algorithm is inadequate to explain the algorithms output. We measured EFR for two algorithms that automatically detect consolidation on chest X-rays to determine the applicability of the metric and observed a lower EFR for the model that had lower sensitivity for identifying consolidation on chest X-rays, implying that trustworthiness of a model should be determined not only by routine statistical metrics but also by novel ‘clinically-oriented’ models.Competing Interest StatementThe authors have declared no competing interest.Funding StatementNo external funding received.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:Since de-identified retrospectively acquired data was used for this study, the IRB named "Mahajan Imaging Institutional Ethics Committee" gave exemption for IRB approval for the same.All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesChest X-Ray data with bounding boxes drawn by AI and humans is available at https://github.com/caringresearch/clinical-explainability-failure-paper/ https://github.com/caringresearch/clinical-explainability-failure-paper/