RT Journal Article
SR Electronic
T1 Development of a Real-Time Risk Model (RTRM) for Predicting In-Hospital COVID-19 Mortality
JF medRxiv
FD Cold Spring Harbor Laboratory Press
SP 2021.04.26.21256138
DO 10.1101/2021.04.26.21256138
A1 Daniel Schlauch
A1 Arielle M. Fisher
A1 Jessica Correia
A1 Xiaotong Fu
A1 Casey Martin
A1 Angela Junglen
A1 Howard A. Burris
A1 Lindsay E. Sears
A1 Gregg Fromell
A1 Mick Correll
A1 Charles F. LeMaistre
A1 Shanna A. Arnold Egloff
YR 2021
UL http://medrxiv.org/content/early/2021/04/28/2021.04.26.21256138.abstract
AB Background With over 83 million cases and 1.8 million deaths reported worldwide by the end of 2020 for SARS-CoV-2 (COVID-19), there is an urgent need to enhance identification of high-risk populations to properly evaluate therapy effectiveness with real-world evidence and improve outcomes.Methods Baseline and daily indicators were evaluated using electronic health records for 46,971 patients hospitalized with COVID-19 from 176 HCA Healthcare-affiliated hospitals, presenting from March to September 2020, to develop a real-time risk model (RTRM) of all-cause, hospitalized mortality. Patient facility, dates-of-care, clinico-demographics, comorbidities, vitals, laboratory markers, and respiratory support findings were aggregated in a logistic regression model.Findings The RTRM predicted overall mortality as well as mortality 1, 3, and 7 days in advance with an area under the receiver operating characteristic curve (AUCROC) of 0.905, 0.911, 0.905, and 0.901 respectively, significantly outperforming a combined model of age and daily modified WHO progression scale (all p<0.0001; AUCROC, 0.846, 0.848, 0.850, and 0.852). The RTRM delineated risk at presentation from ongoing risk associated with medical care and showed that mortality rates decreased over time due to both decreased severity and changes in care.Interpretation To our knowledge, this study is the largest of its kind to comprehensively evaluate predictors and incorporate daily risk measures of COVID-19 mortality. The RTRM validates current literature trends in mortality across time and allows direct translation for research and clinical applications.Evidence before this study Due to the rapidly evolving nature of the COVID-19 pandemic, the body of evidence and published literature was considered prior to study initiation and throughout the course of the study. Although at study initiation there was a growing consensus that age and disease severity at presentation were the greatest contributors to predicting in-hospital mortality, there was less of a consensus on the key demographics, comorbidities, vitals and laboratory values. In addition, early on, most potential predictors of in-hospital mortality had been assessed by univariable analysis. In April of 2020, a systematic review of prediction studies for COVID-19 revealed that there were only 8 publications for prognosis of hospital mortality. All were deemed to have high potential for bias due to low sample size, model overfitting, vague reporting and/or insufficient follow-up. Over the duration of the study, in-hospital prediction models were published ranging from simplified scores to machine learning. There were at least 8 prediction studies that were published during the course of our own that had comparable sample size or extensive multivariable analysis with the greatest accuracy of prediction reported as 74%. Moreover, a report in December of 2020 independently validated 4 simple prediction models, with none achieving greater than an AUCROC of 0.72%. Lastly, an eight-variable score developed by a UK consortium on a comparable sample size demonstrated an AUCROC of 0.77. To our knowledge, however, none to-date have modeled daily risk beyond baseline.We frequently assessed World Health Organization (WHO) resources as well as queried both MedRXIV and PubMed with the search terms “COVID”, “prediction”, “hospital” and “mortality” to ensure we were assessing all potential predictors of hospitalized mortality. The last search was performed on January 5, 2021 with the addition of “multi”, “daily”, “real time” or “longitudinal” terms to confirm the novelty of our study. No date restrictions or language filters were applied.Added value of this study To our knowledge, this study is the largest and most geographically diverse of its kind to comprehensively evaluate predictors of in-hospital COVID-19 mortality that are available retrospectively in electronic health records and to incorporate longitudinal, daily risk measures to create risk trajectories over the entire hospital stay. Not only does our Real-Time Risk Model (RTRM) validate current literature, demonstrating reduced mortality over the course of the COVID-19 pandemic and identifying age and WHO severity as major drivers of mortality in regards to baseline characteristics, but it also outperforms a model of age and daily WHO score combined, achieving an AUCROC of 0.91 on the test set. Furthermore, the fact that the RTRM delineates risk at baseline from risk over the course of care allows more granular interpretation of the impact of various parameters on mortality risk, as demonstrated in the current study using both sex disparity and calendar epochs that were based on evolving treatment recommendations as proofs-of-principle.Implications of all the available evidence The goal of the RTRM was to create a flexible tool that could be used to assess intervention and treatment efficacy in real-world, evidence-based studies as well as provide real-time risk assessment to aid clinical decisions and resourcing with further development. Implications of this work are broad. The depth of the multi-facility, harmonized electronic health record (EHR) dataset coupled with the transparency we provide in the RTRM results provides a resource for others to interpret impact of markers of interest and utilize data that is relevant to their own studies. The RTRM will allow optimal matching in retrospective cohort studies and provide a more granular endpoint for evaluation of interventions beyond general effectiveness, such as optimal delivery, including dosing and timing, and identification of the population/s benefiting from an intervention or combination of interventions. In addition, beyond the scope of the current study, the RTRM and its resultant daily risk scores allow for flexibility in developing prediction models for other clinical outcomes, such as progression of pulmonary disease, need for invasive mechanical ventilation, and development of sepsis and/or multiorgan failure, all of which could provide a framework for real-time personalized care.Competing Interest StatementThe views expressed in this publication represent those of the authors and do not necessarily represent the official views of HCA Healthcare or any of its affiliated entities. None of the authors declared any conflict of interest related to the current study beyond employment with an affiliate of HCA Healthcare. HAB reported grant funding from MedImmune, Boehringer Ingelheim, Merck, Moderna, Verastem, Harpoon, Jounce, Janssen, BIND Therapeutics, Pfizer, Vertex, Gilead, Bayer, Incyte, AstraZeneca, Novartis, Seattle Genetics, GlaxoSmithKline, BioAtla, Agios, BioMed Valley, TG Therapeutics, eFFECTOR, CicloMed, Array, Roche/Genentech, Arvinas, Bristol-Myers Squibb, Macrogenics, CytomX, Arch, Revolution Medicine, Lilly, Tesaro, Takeda/Millennium, miRNA, Kyocera and Foundation Medicine (all paid to his institution); consulting fees from Incyte, AstraZeneca, Celgene, and Forma Therapeutics (all paid to his institution); non-compensated consulting services from Novartis, Bayer, Pfizer, GRAIL and Daiichi Sankyo; expert testimony from Novartis (paid to his institution) and stock ownership in HCA Healthcare. DS, AMF, JC, MC, and SAAE reported stock ownership in HCA Healthcare.Funding StatementThe project was sponsored by HCA Healthcare where all authors are employed by an affiliate. HCA Healthcare prioritized and provided resources for this research and was involved in the decision to submit for publication. Data collection, analysis, and the writing of the report were performed independently by employees. No external funding was received for this work.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:This study was deemed exempt, non-human subjects research by the HCA Healthcare governing institutional review board.All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesThe data that support the findings of this study are available upon request from the corresponding author. The data are not publicly available due to privacy restrictions. Extensive detail on all variables utilized by the RTRM has been provided in the supplemental materials, including significance and effect size.