Abstract
Purpose To advance the usage of CXRs as a viable solution for efficient COVID-19 diagnostics by providing large-scale annotations of the abnormalities in frontal CXRs in BIMCV-COVID19+ database, and to provide a robust evaluation mechanism to facilitate its usage.
Materials and Methods We provide the abnormality annotations in frontal CXRs by creating bounding boxes. The frontal CXRs are a part of the existing BIMCV-COVID19+ database. We also define four different protocols for robust evaluation of semantic segmentation and classification algorithms. Finally, we benchmark the defined protocols and report the results using popular deep learning models as a part of this study.
Results For semantic segmentation, Mask-RCNN performs the best among all the models with a DICE score of 0.43 ± 0.01. For classification, we observe that MobileNetv2 yields the best results for 2-class and 3-class classification. We also observe that deep models report a lower performance for classifying other classes apart from the COVID class.
Conclusion By making the annotated data and protocols available to the scientific community, we aim to advance the usage of CXRs as a viable solution for efficient COVID-19 diagnostics. This large-scale data will be useful for ML algorithms and can be used for learning radiological patterns observed in COVID-19 patients. Further, the protocols will facilitate ML practitioners for unified large-scale evaluation of their algorithms.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
The study is funded by Rakshak Project by IIT-JODHPUR
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
IRB exempted
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
There are two annotation files in this release. Filename - Study-Level_Annotations.csv, contains two types of study level annotations of these X-rays (Normal/Abnormal status & Image Quality). Filename - BoundingBox.csv, conatins pixel level image annotations for 10 Pathologies - Atelectasis, Cardiomegaly, Consolidation/Ground Glass opacity, Edema, Nodule, Pleural Effusion, Pleural Other, Pneumothorax. The corresponding X-rays were released by the Medical Imaging Data Bank of the Valencia region (BIMCV). They can be downloaded at the following link - https://bimcv.cipf.es/bimcv-projects/bimcv-covid19.
Abbreviations
- (COVID-19)
- Coronavirus Disease 2019
- (RT-PCR)
- real time polymerase chain reaction
- (AI)
- artificial intelligence
- (ROC)
- receiver operating characteristic
- (AUC)
- area under the ROC curve
- (CNN)
- convolutional neural network
- (CXR)
- chest x-ray
- (ML)
- machine learning
