Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Development of a Real-Time Risk Model (RTRM) for Predicting In-Hospital COVID-19 Mortality

View ORCID ProfileDaniel Schlauch, View ORCID ProfileArielle M. Fisher, Jessica Correia, Xiaotong Fu, Casey Martin, View ORCID ProfileAngela Junglen, View ORCID ProfileHoward A. Burris, View ORCID ProfileLindsay E. Sears, Gregg Fromell, Mick Correll, View ORCID ProfileCharles F. LeMaistre, View ORCID ProfileShanna A. Arnold Egloff
doi: https://doi.org/10.1101/2021.04.26.21256138
Daniel Schlauch
1HCA Healthcare, Sarah Cannon, USA, 1100 Dr. Martin L. King Jr. Blvd., Suite 800, Nashville, TN 37203
2Genospace, USA, 27 School Street, Boston, MA 02108
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Daniel Schlauch
Arielle M. Fisher
1HCA Healthcare, Sarah Cannon, USA, 1100 Dr. Martin L. King Jr. Blvd., Suite 800, Nashville, TN 37203
2Genospace, USA, 27 School Street, Boston, MA 02108
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Arielle M. Fisher
Jessica Correia
1HCA Healthcare, Sarah Cannon, USA, 1100 Dr. Martin L. King Jr. Blvd., Suite 800, Nashville, TN 37203
2Genospace, USA, 27 School Street, Boston, MA 02108
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Xiaotong Fu
1HCA Healthcare, Sarah Cannon, USA, 1100 Dr. Martin L. King Jr. Blvd., Suite 800, Nashville, TN 37203
2Genospace, USA, 27 School Street, Boston, MA 02108
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Casey Martin
1HCA Healthcare, Sarah Cannon, USA, 1100 Dr. Martin L. King Jr. Blvd., Suite 800, Nashville, TN 37203
3HCA Healthcare, HCA Research Institute (HRI), USA, One Park Plaza, Nashville, TN 37203
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Angela Junglen
1HCA Healthcare, Sarah Cannon, USA, 1100 Dr. Martin L. King Jr. Blvd., Suite 800, Nashville, TN 37203
3HCA Healthcare, HCA Research Institute (HRI), USA, One Park Plaza, Nashville, TN 37203
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Angela Junglen
Howard A. Burris
1HCA Healthcare, Sarah Cannon, USA, 1100 Dr. Martin L. King Jr. Blvd., Suite 800, Nashville, TN 37203
3HCA Healthcare, HCA Research Institute (HRI), USA, One Park Plaza, Nashville, TN 37203
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Howard A. Burris
Lindsay E. Sears
1HCA Healthcare, Sarah Cannon, USA, 1100 Dr. Martin L. King Jr. Blvd., Suite 800, Nashville, TN 37203
3HCA Healthcare, HCA Research Institute (HRI), USA, One Park Plaza, Nashville, TN 37203
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Lindsay E. Sears
Gregg Fromell
1HCA Healthcare, Sarah Cannon, USA, 1100 Dr. Martin L. King Jr. Blvd., Suite 800, Nashville, TN 37203
3HCA Healthcare, HCA Research Institute (HRI), USA, One Park Plaza, Nashville, TN 37203
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Mick Correll
1HCA Healthcare, Sarah Cannon, USA, 1100 Dr. Martin L. King Jr. Blvd., Suite 800, Nashville, TN 37203
2Genospace, USA, 27 School Street, Boston, MA 02108
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Charles F. LeMaistre
1HCA Healthcare, Sarah Cannon, USA, 1100 Dr. Martin L. King Jr. Blvd., Suite 800, Nashville, TN 37203
3HCA Healthcare, HCA Research Institute (HRI), USA, One Park Plaza, Nashville, TN 37203
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Charles F. LeMaistre
Shanna A. Arnold Egloff
1HCA Healthcare, Sarah Cannon, USA, 1100 Dr. Martin L. King Jr. Blvd., Suite 800, Nashville, TN 37203
3HCA Healthcare, HCA Research Institute (HRI), USA, One Park Plaza, Nashville, TN 37203
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Shanna A. Arnold Egloff
  • For correspondence: shanna.arnoldegloff@sarahcannon.com
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

ABSTRACT

Background With over 83 million cases and 1.8 million deaths reported worldwide by the end of 2020 for SARS-CoV-2 (COVID-19), there is an urgent need to enhance identification of high-risk populations to properly evaluate therapy effectiveness with real-world evidence and improve outcomes.

Methods Baseline and daily indicators were evaluated using electronic health records for 46,971 patients hospitalized with COVID-19 from 176 HCA Healthcare-affiliated hospitals, presenting from March to September 2020, to develop a real-time risk model (RTRM) of all-cause, hospitalized mortality. Patient facility, dates-of-care, clinico-demographics, comorbidities, vitals, laboratory markers, and respiratory support findings were aggregated in a logistic regression model.

Findings The RTRM predicted overall mortality as well as mortality 1, 3, and 7 days in advance with an area under the receiver operating characteristic curve (AUCROC) of 0.905, 0.911, 0.905, and 0.901 respectively, significantly outperforming a combined model of age and daily modified WHO progression scale (all p<0.0001; AUCROC, 0.846, 0.848, 0.850, and 0.852). The RTRM delineated risk at presentation from ongoing risk associated with medical care and showed that mortality rates decreased over time due to both decreased severity and changes in care.

Interpretation To our knowledge, this study is the largest of its kind to comprehensively evaluate predictors and incorporate daily risk measures of COVID-19 mortality. The RTRM validates current literature trends in mortality across time and allows direct translation for research and clinical applications.

Evidence before this study Due to the rapidly evolving nature of the COVID-19 pandemic, the body of evidence and published literature was considered prior to study initiation and throughout the course of the study. Although at study initiation there was a growing consensus that age and disease severity at presentation were the greatest contributors to predicting in-hospital mortality, there was less of a consensus on the key demographics, comorbidities, vitals and laboratory values. In addition, early on, most potential predictors of in-hospital mortality had been assessed by univariable analysis. In April of 2020, a systematic review of prediction studies for COVID-19 revealed that there were only 8 publications for prognosis of hospital mortality. All were deemed to have high potential for bias due to low sample size, model overfitting, vague reporting and/or insufficient follow-up. Over the duration of the study, in-hospital prediction models were published ranging from simplified scores to machine learning. There were at least 8 prediction studies that were published during the course of our own that had comparable sample size or extensive multivariable analysis with the greatest accuracy of prediction reported as 74%. Moreover, a report in December of 2020 independently validated 4 simple prediction models, with none achieving greater than an AUCROC of 0.72%. Lastly, an eight-variable score developed by a UK consortium on a comparable sample size demonstrated an AUCROC of 0.77. To our knowledge, however, none to-date have modeled daily risk beyond baseline.

We frequently assessed World Health Organization (WHO) resources as well as queried both MedRXIV and PubMed with the search terms “COVID”, “prediction”, “hospital” and “mortality” to ensure we were assessing all potential predictors of hospitalized mortality. The last search was performed on January 5, 2021 with the addition of “multi”, “daily”, “real time” or “longitudinal” terms to confirm the novelty of our study. No date restrictions or language filters were applied.

Added value of this study To our knowledge, this study is the largest and most geographically diverse of its kind to comprehensively evaluate predictors of in-hospital COVID-19 mortality that are available retrospectively in electronic health records and to incorporate longitudinal, daily risk measures to create risk trajectories over the entire hospital stay. Not only does our Real-Time Risk Model (RTRM) validate current literature, demonstrating reduced mortality over the course of the COVID-19 pandemic and identifying age and WHO severity as major drivers of mortality in regards to baseline characteristics, but it also outperforms a model of age and daily WHO score combined, achieving an AUCROC of 0.91 on the test set. Furthermore, the fact that the RTRM delineates risk at baseline from risk over the course of care allows more granular interpretation of the impact of various parameters on mortality risk, as demonstrated in the current study using both sex disparity and calendar epochs that were based on evolving treatment recommendations as proofs-of-principle.

Implications of all the available evidence The goal of the RTRM was to create a flexible tool that could be used to assess intervention and treatment efficacy in real-world, evidence-based studies as well as provide real-time risk assessment to aid clinical decisions and resourcing with further development. Implications of this work are broad. The depth of the multi-facility, harmonized electronic health record (EHR) dataset coupled with the transparency we provide in the RTRM results provides a resource for others to interpret impact of markers of interest and utilize data that is relevant to their own studies. The RTRM will allow optimal matching in retrospective cohort studies and provide a more granular endpoint for evaluation of interventions beyond general effectiveness, such as optimal delivery, including dosing and timing, and identification of the population/s benefiting from an intervention or combination of interventions. In addition, beyond the scope of the current study, the RTRM and its resultant daily risk scores allow for flexibility in developing prediction models for other clinical outcomes, such as progression of pulmonary disease, need for invasive mechanical ventilation, and development of sepsis and/or multiorgan failure, all of which could provide a framework for real-time personalized care.

Competing Interest Statement

The views expressed in this publication represent those of the authors and do not necessarily represent the official views of HCA Healthcare or any of its affiliated entities. None of the authors declared any conflict of interest related to the current study beyond employment with an affiliate of HCA Healthcare. HAB reported grant funding from MedImmune, Boehringer Ingelheim, Merck, Moderna, Verastem, Harpoon, Jounce, Janssen, BIND Therapeutics, Pfizer, Vertex, Gilead, Bayer, Incyte, AstraZeneca, Novartis, Seattle Genetics, GlaxoSmithKline, BioAtla, Agios, BioMed Valley, TG Therapeutics, eFFECTOR, CicloMed, Array, Roche/Genentech, Arvinas, Bristol-Myers Squibb, Macrogenics, CytomX, Arch, Revolution Medicine, Lilly, Tesaro, Takeda/Millennium, miRNA, Kyocera and Foundation Medicine (all paid to his institution); consulting fees from Incyte, AstraZeneca, Celgene, and Forma Therapeutics (all paid to his institution); non-compensated consulting services from Novartis, Bayer, Pfizer, GRAIL and Daiichi Sankyo; expert testimony from Novartis (paid to his institution) and stock ownership in HCA Healthcare. DS, AMF, JC, MC, and SAAE reported stock ownership in HCA Healthcare.

Funding Statement

The project was sponsored by HCA Healthcare where all authors are employed by an affiliate. HCA Healthcare prioritized and provided resources for this research and was involved in the decision to submit for publication. Data collection, analysis, and the writing of the report were performed independently by employees. No external funding was received for this work.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

This study was deemed exempt, non-human subjects research by the HCA Healthcare governing institutional review board.

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Data Availability

The data that support the findings of this study are available upon request from the corresponding author. The data are not publicly available due to privacy restrictions. Extensive detail on all variables utilized by the RTRM has been provided in the supplemental materials, including significance and effect size.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted April 28, 2021.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Development of a Real-Time Risk Model (RTRM) for Predicting In-Hospital COVID-19 Mortality
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Development of a Real-Time Risk Model (RTRM) for Predicting In-Hospital COVID-19 Mortality
Daniel Schlauch, Arielle M. Fisher, Jessica Correia, Xiaotong Fu, Casey Martin, Angela Junglen, Howard A. Burris, Lindsay E. Sears, Gregg Fromell, Mick Correll, Charles F. LeMaistre, Shanna A. Arnold Egloff
medRxiv 2021.04.26.21256138; doi: https://doi.org/10.1101/2021.04.26.21256138
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Development of a Real-Time Risk Model (RTRM) for Predicting In-Hospital COVID-19 Mortality
Daniel Schlauch, Arielle M. Fisher, Jessica Correia, Xiaotong Fu, Casey Martin, Angela Junglen, Howard A. Burris, Lindsay E. Sears, Gregg Fromell, Mick Correll, Charles F. LeMaistre, Shanna A. Arnold Egloff
medRxiv 2021.04.26.21256138; doi: https://doi.org/10.1101/2021.04.26.21256138

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Infectious Diseases (except HIV/AIDS)
Subject Areas
All Articles
  • Addiction Medicine (269)
  • Allergy and Immunology (552)
  • Anesthesia (135)
  • Cardiovascular Medicine (1752)
  • Dentistry and Oral Medicine (238)
  • Dermatology (172)
  • Emergency Medicine (312)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (657)
  • Epidemiology (10792)
  • Forensic Medicine (8)
  • Gastroenterology (586)
  • Genetic and Genomic Medicine (2939)
  • Geriatric Medicine (287)
  • Health Economics (532)
  • Health Informatics (1921)
  • Health Policy (835)
  • Health Systems and Quality Improvement (744)
  • Hematology (291)
  • HIV/AIDS (628)
  • Infectious Diseases (except HIV/AIDS) (12510)
  • Intensive Care and Critical Care Medicine (687)
  • Medical Education (299)
  • Medical Ethics (86)
  • Nephrology (324)
  • Neurology (2791)
  • Nursing (151)
  • Nutrition (432)
  • Obstetrics and Gynecology (557)
  • Occupational and Environmental Health (597)
  • Oncology (1460)
  • Ophthalmology (443)
  • Orthopedics (172)
  • Otolaryngology (255)
  • Pain Medicine (190)
  • Palliative Medicine (56)
  • Pathology (380)
  • Pediatrics (865)
  • Pharmacology and Therapeutics (363)
  • Primary Care Research (336)
  • Psychiatry and Clinical Psychology (2636)
  • Public and Global Health (5350)
  • Radiology and Imaging (1009)
  • Rehabilitation Medicine and Physical Therapy (595)
  • Respiratory Medicine (726)
  • Rheumatology (329)
  • Sexual and Reproductive Health (289)
  • Sports Medicine (278)
  • Surgery (327)
  • Toxicology (47)
  • Transplantation (149)
  • Urology (125)