PT  - JOURNAL ARTICLE
AU  - Hassan, Doaa
AU  - Gill, Hunter Mathias
AU  - Happe, Michael
AU  - Bhatwadekar, Ashay D.
AU  - Hajrasouliha, Amir R.
AU  - Janga, Sarath Chandra
TI  - Combining Transfer Learning with Retinal Lesions Features for Accurate Detection of Diabetic Retinopathy
AID  - 10.1101/2022.09.23.22280273
DP  - 2022 Jan 01
TA  - medRxiv
PG  - 2022.09.23.22280273
4099  - http://medrxiv.org/content/early/2022/09/25/2022.09.23.22280273.short
4100  - http://medrxiv.org/content/early/2022/09/25/2022.09.23.22280273.full
AB  - Diabetic retinopathy (DR) is a late microvascular complication of Diabetes Mellitus (DM) that could lead to permanent blindness in patients, without early detection. Although adequate management of DM via regular eye examination can preserve vision in in 98% of the DR cases, DR screening and diagnoses based on clinical lesion features devised by expert clinicians; are costly, time-consuming and not sufficiently accurate. This raises the requirements for Artificial Intelligent (AI) systems which can accurately detect DR automatically and thus preventing DR before affecting vision. Hence, such systems can help clinician experts in certain cases and aid ophthalmologists in rapid diagnoses. To address such requirements, several approaches have been proposed in the literature that use Machine Learning (ML) and Deep Learning (DL) techniques to develop such systems. However, these approaches ignore the highly valuable clinical lesion features that could contribute significantly to the accurate detection of DR. Therefore, in this study we introduce a framework called DR-detector that employs the Extreme Gradient Boosting (XGBoost) ML model trained via the combination of the features extracted by the pretrained convolutional neural networks commonly known as transfer learning (TL) models and the clinical retinal lesion features for accurate detection of DR. The retinal lesion features are extracted via image segmentation technique using the UNET DL model and captures exudates (EXs), microaneurysms (MAs), and hemorrhages (HEMs) that are relevant lesions for DR detection. The feature combination approach implemented in DR-detector has been applied to two common TL models in the literature namely VGG-16 and ResNet-50. We trained the DR-detector model using a training dataset comprising of 1840 color fundus images collected from e-ophtha, retinal lesions and APTOS 2019 Kaggle datasets of which 920 images are healthy. To validate the DR-detector model, we test the model on external dataset that consists of 81 healthy images collected from High-Resolution Fundus (HRF) dataset and MESSIDOR-2 datasets and 81 images with DR signs collected from Indian Diabetic Retinopathy Image Dataset (IDRID) dataset annotated for DR by expert. The experimental results show that the DR-detector model achieves a testing accuracy of 100% in detecting DR after training it with the combination of ResNet-50 and lesion features and 99.38% accuracy after training it with the combination of VGG-16 and lesion features. More importantly, the results also show a higher contribution of specific lesion features toward the performance of the DR-detector model. For instance, using only the hemorrhages feature to train the model, our model achieves an accuracy of 99.38 in detecting DR, which is higher than the accuracy when training the model with the combination of all lesion features (89%) and equal to the accuracy when training the model with the combination of all lesions and VGG-16 features together. This highlights the possibility of using only the clinical features, such as lesions that are clinically interpretable, to build the next generation of robust artificial intelligence (AI) systems with great clinical interpretability for DR detection. The code of the DR-detector framework is available on GitHub at https://github.com/Janga-Lab/DR-detector and can be readily employed for detecting DR from retinal image datasets.Competing Interest StatementThe authors have declared no competing interest.Clinical Protocols https://www.frontiersin.org/journals/medicine Funding StatementThis research was funded by the National Eye Institute of the NIH under Award Number R01EY032080 (AB, AH AJ, and SJ) and a pilot grant IUPUI Institute of Integrative Artificial Intelligence (iAI) (SCJ and AJ). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:We used a training dataset comprising of 1840 color fundus images collected from e-ophtha, retinal lesions and APTOS 2019 Kaggle datasets of which 920 images are healthy. We also used testing external dataset that consists of 81 healthy images collected from High-Resolution Fundus (HRF) dataset and MESSIDOR-2 datasets and 81 images with DR signs collected from Indian Diabetic Retinopathy Image Dataset (IDRID) dataset annotated for DR by expert.I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesAll data produced are available online at: 1. https://www.adcis.net/en/third-party/e-ophtha/ 2. https://github.com/WeiQijie/retinal-lesions 3. https://www.kaggle.com/c/aptos2019-blindness-detection/data 4. https://www5.cs.fau.de/research/data/fundus-images/ 5. https://idrid.grand-challenge.org 6. https://www.adcis.net/en/third-party/messidor2/ https://github.com/Janga-Lab/DR-detector https://www.adcis.net/en/third-party/e-ophtha/ https://github.com/WeiQijie/retinal-lesions https://www.kaggle.com/c/aptos2019-blindness-detection/data https://www5.cs.fau.de/research/data/fundus-images/ https://idrid.grand-challenge.org https://www.adcis.net/en/third-party/messidor2/