PT - JOURNAL ARTICLE AU - Stephen S.F. Yip AU - Zan Klanecek AU - Shotaro Naganawa AU - John Kim AU - Andrej Studen AU - Luciano Rivetti AU - Robert Jeraj TI - Performance and Robustness of Machine Learning-based Radiomic COVID-19 Severity Prediction AID - 10.1101/2020.09.07.20189977 DP - 2020 Jan 01 TA - medRxiv PG - 2020.09.07.20189977 4099 - http://medrxiv.org/content/early/2020/09/09/2020.09.07.20189977.short 4100 - http://medrxiv.org/content/early/2020/09/09/2020.09.07.20189977.full AB - Objectives This study investigated the performance and robustness of radiomics in predicting COVID-19 severity in a large public cohort.Methods A public dataset of 1110 COVID-19 patients (1 CT/patient) was used. Using CTs and clinical data, each patient was classified into mild, moderate, and severe by two observers: (1) dataset provider and (2) a board-certified radiologist. For each CT, 107 radiomic features were extracted. The dataset was randomly divided into a training (60%) and holdout validation (40%) set. During training, features were selected and combined into a logistic regression model for predicting severe cases from mild and moderate cases. The models were trained and validated on the classifications by both observers. AUC quantified the predictive power of models. To determine model robustness, the trained models was cross-validated on the inter-observer’s classifications.Results A single feature alone was sufficient to predict mild from severe COVID-19 with and (p<< 0.01). The most predictive features were the distribution of small size-zones (GLSZM-SmallAreaEmphasis) for provider’s classification and linear dependency of neighboring voxels (GLCM-Correlation) for radiologist’s classification. Cross-validation showed that both . In predicting moderate from severe COVID-19, first-order-Median alone had sufficient predictive power of . For radiologist’s classification, the predictive power of the model increased to as the number of features grew from 1 to 5. Cross-validation yielded and .Conclusions Radiomics significantly predicted different levels of COVID-19 severity. The prediction was moderately sensitive to inter-observer classifications, and thus need to be used with caution.Key pointsInterpretable radiomic features can predict different levels of COVID-19 severityMachine Learning-based radiomic models were moderately sensitive to inter-observer classifications, and thus need to be used with cautionCompeting Interest StatementS.S.F.Y. and R.J. of this manuscript declare relationships with AIQ Solutions Inc: S.S.F.Y. is an employee and a shareholder of AIQ Solutions, Inc. R.J. is a consultant and shareholder of AIQ Solutions, Inc.Funding StatementThe authors state that this work has not received any funding.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:Institutional Review Board approval was not required because public MosMedData dataset was used. This public dataset is freely available for scientific research on https://mosmed.aiAll necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesThis public dataset is freely available for scientific research on https://mosmed.ai https://mosmed.ai COVID-19Coronavirus disease 2019GLCMGray level co-occurrence matrixGLDMGray level dependence matrixGLSZMGray level size zone matrixIMC2Informational measure of correlation 2MLMachine learningMRMRMaximum relevance and minimum redundancyRFERecursive feature elimination