Abstract
Objectives Existing UK prognostic models for patients admitted to hospital with COVID-19 are limited by reliance on comorbidities, which are under-recorded in secondary care, and lack of imaging data among the candidate predictors. Our aims were to develop and externally validate novel prognostic models for adverse outcomes (death, intensive therapy unit (ITU) admission) in UK secondary care; and externally validate the existing 4C score.
Design Candidate predictors included demographic variables, symptoms, physiological measures, imaging, laboratory tests. Final models used logistic regression with stepwise selection.
Setting Model development was performed in data from University Hospitals Birmingham (UHB). External validation was performed in the CovidCollab dataset.
Participants Patients with COVID-19 admitted to UHB January-August 2020 were included.
Main outcome measures Death and ITU admission within 28 days of admission.
Results 1040 patients with COVID-19 were included in the derivation cohort; 288 (28%) died and 183 (18%) were admitted to ITU within 28 days of admission. Area under the receiver operating curve (AUROC) for mortality was 0.791 (95%CI 0.761-0.822) in UHB and 0.767 (95%CI 0.754-0.780) in CovidCollab; AUROC for ITU admission was 0.906 (95%CI 0.883-0.929) in UHB and 0.811 (95%CI 0.795-0.828) in CovidCollab. Models showed good calibration. Addition of comorbidities to candidate predictors did not improve model performance. AUROC for the 4C score in the UHB dataset was 0.754 (95%CI 0.721-0.786).
Conclusions The novel prognostic models showed good discrimination and calibration in derivation and external validation datasets, and outperformed the existing 4C score. The models can be integrated into electronic medical records systems to calculate each individual patient’s probability of death or ITU admission at the time of hospital admission. Implementation of the models and clinical utility should be evaluated.
Strengths and limitations of this study
We developed novel prognostic models predicting mortality and ITU admission within 28 days of admission for patients hospitalised with COVID-19, using a large routinely collected dataset gathered at admission with a wide range of possible predictors (demographic variables, symptoms, physiological measures, imaging, laboratory test results).
These novel models showed good discrimination and calibration in both derivation and external validation cohorts, and outperformed the existing ISARIC model and 4C score in the derivation dataset. We found that addition of comorbidities to the set of candidate predictors included in model derivation did not improve model performance.
If integrated into hospital electronic medical records systems, the model algorithms will provide a predicted probability of mortality or ITU admission for each patient based on their individual data at, or close to, the time of admission, which will support clinicians’ decision making with regard to appropriate patient care pathways and triage. This information might also assist clinicians in explaining complex prognostic assessments and decisions to patients and their relatives.
A limitation of the study was that in the external validation cohort we were unable to examine all of the predictors included in the original full UHB model due to only a reduced set of candidate predictors being available in CovidCollab. Nevertheless, the reduced model performed well and the results suggest it may be applicable in a wide range of datasets where only a reduced set of predictor variables is available.
Furthermore, it was not possible to carry out stratified analysis by ethnicity as the UHB dataset contained too few patients in most of the strata, and no ethnicity data was available in the CovidCollab dataset.
Competing Interest Statement
NJA, ES, KN, MP, AD, CS, TT and YT report a grant from UKRI MRC during the conduct of the study. ES reports grants from National Institute for Health Research (NIHR), Wellcome Trust, MRC, Health Data Research UK (HDR-UK), British Lung Foundation, and Alpha 1 Foundation outside the submitted work. KN reports grants from MRC and HDR-UK outside the submitted work. DP reports grants from NIHR, MRC, and Chernakovsky Foundation outside the submitted work. All other authors have nothing to declare.
Funding Statement
This work was funded by the Medical Research Council UK Research and Innovation (MRC UKRI) (reference COV0306). The funder had no role in developing the research question or the study protocol.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Ethical approval was provided by the East Midlands Derby REC (reference: 20/EM/0158) for the PIONEER Research Database (data from University Hospitals Birmingham). For CovidCollab data, local, regional, and national approvals were obtained from all participating sites. In the UK, this study was registered as clinical audit or service evaluation, with approval granted in line with local information governance policies, in line with assessment and guidance by the Health Research Authority. At the lead site (University Hospitals Birmingham NHS Trust) this study was registered as clinical audit (CARMS-15986). In other countries, local principal investigators were responsible for obtaining approvals in line with their local, regional, and national guidelines and recommendations. Only routinely collected data was collected and patient care was not altered by this study. Anonymised data was securely transferred to the Birmingham Centre for Prospective and Observational Studies (BiCOPS), University of Birmingham via REDCap. All sites were required to confirm that approvals were in place prior to being provided with logins; written data sharing agreements were arranged where requested by individual sites.
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
No additional data available.
