Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Clinical Validation of Saliency Maps for Understanding Deep Neural Networks in Ophthalmology

View ORCID ProfileMurat Seçkin Ayhan, Louis Benedikt Kümmerle, Laura Kühlewein, Werner Inhoffen, Gulnar Aliyeva, Focke Ziemssen, Philipp Berens
doi: https://doi.org/10.1101/2021.05.05.21256683
Murat Seçkin Ayhan
1Institute for Ophthalmic Research, University of Tübingen, Tübingen, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Murat Seçkin Ayhan
  • For correspondence: murat-seckin.ayhan@uni-tuebingen.de
Louis Benedikt Kümmerle
1Institute for Ophthalmic Research, University of Tübingen, Tübingen, Germany
2Institute of Computational Biology, Helmholtz Center Munich, Munich, Germany
3Institute for Tissue Engineering and Regenerative Medicine (iTERM), Helmholtz Center Munich, Munich, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Laura Kühlewein
1Institute for Ophthalmic Research, University of Tübingen, Tübingen, Germany
4University Eye Clinic, University of Tübingen, Tübingen, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Werner Inhoffen
4University Eye Clinic, University of Tübingen, Tübingen, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Gulnar Aliyeva
4University Eye Clinic, University of Tübingen, Tübingen, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Focke Ziemssen
4University Eye Clinic, University of Tübingen, Tübingen, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Philipp Berens
1Institute for Ophthalmic Research, University of Tübingen, Tübingen, Germany
5Department of Computer Science, University of Tübingen, Tübingen, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

Abstract

Deep neural networks (DNNs) have achieved physician-level accuracy on many imaging-based medical diagnostic tasks, for example classification of retinal images in ophthalmology. However, their decision mechanisms are often considered impenetrable leading to a lack of trust by clinicians and patients. To alleviate this issue, a range of explanation methods have been proposed to expose the inner workings of DNNs leading to their decisions. For imaging-based tasks, this is often achieved via saliency maps. The quality of these maps are typically evaluated via perturbation analysis without experts involved. To facilitate the adoption and success of such automated systems, however, it is crucial to validate saliency maps against clinicians. In this study, we used two different network architectures and developed ensembles of DNNs to detect diabetic retinopathy and neovascular age-related macular degeneration from retinal fundus images and optical coherence tomography scans, respectively. We used a variety of explanation methods and obtained a comprehensive set of saliency maps for explaining the ensemble-based diagnostic decisions. Then, we systematically validated saliency maps against clinicians through two main analyses — a direct comparison of saliency maps with the expert annotations of disease-specific pathologies and perturbation analyses using also expert annotations as saliency maps. We found the choice of DNN architecture and explanation method to significantly influence the quality of saliency maps. Guided Backprop showed consistently good performance across disease scenarios and DNN architectures, suggesting that it provides a suitable starting point for explaining the decisions of DNNs on retinal images.

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

This research was supported by the German Ministry of Science and Education (BMBF, 01GQ1601 and 01IS18039A) and the German Science Foundation (BE5601/4-1 and EXC 2064, project number 390727645). Additional funding was provided by Novartis AG through a research grant. The funders did not have any influence in the study planning and design.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The use of the optical coherence tomography scans was permitted by the Institutional Ethics Committee of the University of Tuebingen and was performed in line with all relevant laws and regulations.

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Data Availability

Retinal fundus images used in this study were obtained from publicly available repositories indicated in the manuscript. The optical coherence tomography scans were obtained from the University Eye Clinic and their use was permitted by the Institutional Ethics Committee of the University of Tuebingen.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted May 09, 2021.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Clinical Validation of Saliency Maps for Understanding Deep Neural Networks in Ophthalmology
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Clinical Validation of Saliency Maps for Understanding Deep Neural Networks in Ophthalmology
Murat Seçkin Ayhan, Louis Benedikt Kümmerle, Laura Kühlewein, Werner Inhoffen, Gulnar Aliyeva, Focke Ziemssen, Philipp Berens
medRxiv 2021.05.05.21256683; doi: https://doi.org/10.1101/2021.05.05.21256683
Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
Citation Tools
Clinical Validation of Saliency Maps for Understanding Deep Neural Networks in Ophthalmology
Murat Seçkin Ayhan, Louis Benedikt Kümmerle, Laura Kühlewein, Werner Inhoffen, Gulnar Aliyeva, Focke Ziemssen, Philipp Berens
medRxiv 2021.05.05.21256683; doi: https://doi.org/10.1101/2021.05.05.21256683

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Ophthalmology
Subject Areas
All Articles
  • Addiction Medicine (160)
  • Allergy and Immunology (413)
  • Anesthesia (90)
  • Cardiovascular Medicine (855)
  • Dentistry and Oral Medicine (159)
  • Dermatology (97)
  • Emergency Medicine (248)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (393)
  • Epidemiology (8550)
  • Forensic Medicine (4)
  • Gastroenterology (383)
  • Genetic and Genomic Medicine (1744)
  • Geriatric Medicine (167)
  • Health Economics (371)
  • Health Informatics (1237)
  • Health Policy (619)
  • Health Systems and Quality Improvement (467)
  • Hematology (196)
  • HIV/AIDS (372)
  • Infectious Diseases (except HIV/AIDS) (10281)
  • Intensive Care and Critical Care Medicine (552)
  • Medical Education (192)
  • Medical Ethics (51)
  • Nephrology (210)
  • Neurology (1670)
  • Nursing (97)
  • Nutrition (248)
  • Obstetrics and Gynecology (325)
  • Occupational and Environmental Health (450)
  • Oncology (926)
  • Ophthalmology (263)
  • Orthopedics (100)
  • Otolaryngology (172)
  • Pain Medicine (112)
  • Palliative Medicine (40)
  • Pathology (252)
  • Pediatrics (534)
  • Pharmacology and Therapeutics (247)
  • Primary Care Research (207)
  • Psychiatry and Clinical Psychology (1764)
  • Public and Global Health (3832)
  • Radiology and Imaging (622)
  • Rehabilitation Medicine and Physical Therapy (319)
  • Respiratory Medicine (520)
  • Rheumatology (207)
  • Sexual and Reproductive Health (165)
  • Sports Medicine (157)
  • Surgery (190)
  • Toxicology (36)
  • Transplantation (101)
  • Urology (74)