PT  - JOURNAL ARTICLE
AU  - Jonatan Attergrim
AU  - Kelvin Szolnoky
AU  - Lovisa Strömmer
AU  - Olof Brattström
AU  - Gunilla Whilke
AU  - Martin Jacobsson
AU  - Martin Gerdin Wärnberg
TI  - Predicting Opportunities for Improvement in Trauma Using Machine Learning: A Registry based Study
AID  - 10.1101/2023.01.19.23284654
DP  - 2023 Jan 01
TA  - medRxiv
PG  - 2023.01.19.23284654
4099  - http://medrxiv.org/content/early/2023/01/30/2023.01.19.23284654.short
4100  - http://medrxiv.org/content/early/2023/01/30/2023.01.19.23284654.full
AB  - Rationale The multidisciplinary mortality and morbidity conference is the core of programs that aim to improve the quality of trauma care and is used to identify and address opportunities for improvement based on reviewing patient cases. Current systems rely on audit filters for review selection, a process that is hampered by high frequencies of false positives.Objectives To develop, validate, and compare the performance of different machine learning models for predicting opportunities for improvement.Methods We conducted a registry based study using all patients from the Karolinska university hospital that had been reviewed regarding the presence of opportunity for improvement, a binary consensus decision from the mortality and morbidity conference. We developed eight binary classification models using 45 predictors. Training used an 80%-20% train-test split and 1000 resamples without replacement estimated confidence intervals. Performance (sensitivity, specificity, integrated calibration index, Area under the receiver operating characteristics curve) was also compared to current audit filters.Measurements and Main Results The dataset included 6310 patients where opportunities for improvement were present among 431 (7%) patients. The audit filters (Area under the receiver operating characteristics curve: 0.624) was outperformed by all machine learning models. The best performing model was LightGBM (Area under the receiver operating characteristics curve: 0.789).Conclusions Machine learning models outperform the currently used audit filters and could prove to be valuable additions in the screening for opportunities for improvement. More research is needed on how to increase model performance and how to incorporate these models into trauma quality improvement programs.Impact Our research explores a novel system that predicts opportunities for improvement in trauma patients using machine learning and outperforms the established approach using audit filters. This could allow for reallocation and optimization of review resources as well as a means of possibly identifying new types of opportunities for improvement. The methodology uses state-of-the-art machine learning largely unexplored in a medical context adding to a broader general scientific knowledge reaching outside this project.Competing Interest StatementThe authors have declared no competing interest.Funding StatementSupported by the Swedish Society of Medicine, grant number SLS-973387, and by The Swedish Carnegie Hero Fund.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:The study was approved by Stockholm Research Ethics Review Board, approval number 2021-02541 and 2021-03531.I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesThe data that support the findings of this study are available following the approval of a project suggesting to use the data by the Swedish Ethical Review Authority and the appropriate bodies at the Karolinska University Hospital. More information is available on request from the corresponding author, J. Attergrim.