Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Risk prediction for poor outcome and death in hospital in-patients with COVID-19: derivation in Wuhan, China and external validation in London, UK

Huayu Zhang, Ting Shi, Xiaodong Wu, Xin Zhang, Kun Wang, View ORCID ProfileDaniel Bean, Richard Dobson, View ORCID ProfileJames T Teo, Jiaxing Sun, Pei Zhao, Chenghong Li, Kevin Dhaliwal, Honghan Wu, Qiang Li, Bruce Guthrie
doi: https://doi.org/10.1101/2020.04.28.20082222
Huayu Zhang
1Centre for Medical Informatics, Usher Institute, University of Edinburgh, Scotland, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ting Shi
2Centre for Global Health, Usher Institute, University of Edinburgh, Scotland, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Xiaodong Wu
3Department of Pulmonary and Critical Care Medicine, Shanghai East Hospital, Tongji University, Shanghai, China
4Department of Pulmonary and Critical Care Medicine, Taikang Tongji Hospital, Wuhan, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Xin Zhang
5Department of Pulmonary and Critical Care Medicine, People’s Liberation Army Joint Logistic Support Force 920th Hospital, Yunnan, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kun Wang
3Department of Pulmonary and Critical Care Medicine, Shanghai East Hospital, Tongji University, Shanghai, China
6Department of Pulmonary and Critical Care Medicine, Shanghai General Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Daniel Bean
7Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, England, United Kingdom
8Health Data Research UK London, University College London, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Daniel Bean
Richard Dobson
7Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, England, United Kingdom
8Health Data Research UK London, University College London, London, UK
9Institute of Health Informatics, University College London, London, England, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
James T Teo
10Department of Stroke and Neurology, King’s College Hospital NHS Foundation Trust, London, England, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for James T Teo
Jiaxing Sun
3Department of Pulmonary and Critical Care Medicine, Shanghai East Hospital, Tongji University, Shanghai, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Pei Zhao
3Department of Pulmonary and Critical Care Medicine, Shanghai East Hospital, Tongji University, Shanghai, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Chenghong Li
11Department of Pulmonary and Critical Care Medicine, Wuhan Sixth Hospital, Jianghan University, Wuhan, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kevin Dhaliwal
12Centre for Inflammation Research, Queens Medical Research Institute, University of Edinburgh, Scotland, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Honghan Wu
1Centre for Medical Informatics, Usher Institute, University of Edinburgh, Scotland, United Kingdom
13School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, China
14Health Data Research UK London, University of Edinburgh, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Qiang Li
3Department of Pulmonary and Critical Care Medicine, Shanghai East Hospital, Tongji University, Shanghai, China
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Bruce Guthrie
15Centre for Population Health Sciences, Usher Institute, University of Edinburgh, Scotland, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: bruce.guthrie@ed.ac.uk
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

Abstract

Background Accurate risk prediction of clinical outcome would usefully inform clinical decisions and intervention targeting in COVID-19. The aim of this study was to derive and validate risk prediction models for poor outcome and death in adult inpatients with COVID-19.

Methods Model derivation using data from Wuhan, China used logistic regression with death and poor outcome (death or severe disease) as outcomes. Predictors were demographic, comorbidity, symptom and laboratory test variables. The best performing models were externally validated in data from London, UK.

Findings 4.3% of the derivation cohort (n=775) died and 9.7% had a poor outcome, compared to 34.1% and 42.9% of the validation cohort (n=226). In derivation, prediction models based on age, sex, neutrophil count, lymphocyte count, platelet count, C-reactive protein and creatinine had excellent discrimination (death c-index=0.91, poor outcome c-index=0.88), with good-to-excellent calibration. Using two cut-offs to define low, high and very-high risk groups, derivation patients were stratified in groups with observed death rates of 0.34%, 15.0% and 28.3% and poor outcome rates 0.63%, 8.9% and 58.5%. External validation discrimination was good (c-index death=0.74, poor outcome=0.72) as was calibration. However, observed rates of death were 16.5%, 42.9% and 58.4% and poor outcome 26.3%, 28.4% and 64.8% in predicted low, high and very-high risk groups.

Interpretation Our prediction model using demography and routinely-available laboratory tests performed very well in internal validation in the lower-risk derivation population, but less well in the much higher-risk external validation population. Further external validation is needed. Collaboration to create larger derivation datasets, and to rapidly externally validate all proposed prediction models in a range of populations is needed, before routine implementation of any risk prediction tool in clinical care.

Funding MRC, Wellcome Trust, HDR-UK, LifeArc, participating hospitals, NNSFC, National Key R&D Program, Pudong Health and Family Planning Commission

Evidence before this study Several prognostic models for predicting mortality risk, progression to severe disease, or length of hospital stay in COVID-19 have been published.1 Commonly reported predictors of severe prognosis in patients with COVID-19 include age, sex, computed tomography scan features, C-reactive protein (CRP), lactic dehydrogenase, and lymphocyte count. Symptoms (notably dyspnoea) and comorbidities (e.g. chronic lung disease, cardiovascular disease and hypertension) are also reported to have associations with poor prognosis.2 However, most studies have not described the study population or intended use of prediction models, and external validation is rare and to date done using datasets originating from different Wuhan hospitals.3 Given different patterns of testing and organisation of healthcare pathways, external validation in datasets from other countries is required.

Added value of this study This study used data from Wuhan, China to derive and internally validate multivariable models to predict poor outcome and death in COVID-19 patients after hospital admission, with external validation using data from King’s College Hospital, London, UK. Mortality and poor outcome occurred in 4.3% and 9.7% of patients in Wuhan, compared to 34.1% and 42.9% of patients in London. Models based on age, sex and simple routinely available laboratory tests (lymphocyte count, neutrophil count, platelet count, CRP and creatinine) had good discrimination and calibration in internal validation, but performed only moderately well in external validation. Models based on age, sex, symptoms and comorbidity were adequate in internal validation for poor outcome (ICU admission or death) but had poor performance for death alone.

Implications of all the available evidence This study and others find that relatively simple risk prediction models using demographic, clinical and laboratory data perform well in internal validation but at best moderately in external validation, either because derivation and external validation populations are small (Xie et al3) and/or because they vary greatly in casemix and severity (our study). There are three decision points where risk prediction may be most useful: (1) deciding who to test; (2) deciding which patients in the community are at high-risk of poor outcomes; and (3) identifying patients at high-risk at the point of hospital admission. Larger studies focusing on particular decision points, with rapid external validation in multiple datasets are needed. A key gap is risk prediction tools for use in community triage (decisions to admit, or to keep at home with varying intensities of follow-up including telemonitoring) or in low income settings where laboratory tests may not be routinely available at the point of decision-making. This requires systematic data collection in community and low-income settings to derive and evaluate appropriate models.

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

HW and HZ are supported by Medical Research Council and Health Data Research UK Grant (MR/S004149/1), Industrial Strategy Challenge Grant (MC_PC_18029) and Wellcome Institutional Translation Partnership Award (PIII054). RD is supported by the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London. DMB is funded by a UKRI Innovation Fellowship as part of Health Data Research UK MR/S00310X/1 (https://www.hdruk.ac.uk). KD is supported by LifeArc STOPCOVID award. This work uses data provided by patients and collected by the NHS as part of their care and support. XW is supported by National Natural Science Foundation of China (grant number:81700006). QL is supported by National Key R&D Program (2018YFC1313700), National Natural Science Foundation of China (grant number: 81870064) and the “Gaoyuan” project of Pudong Health and Family Planning Commission (PWYgy2018-06). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health and Social Care. We acknowledge Lingyu Ran and Yongsheng Du for their contribution in data collection.

Author Declarations

All relevant ethical guidelines have been followed; any necessary IRB and/or ethics committee approvals have been obtained and details of the IRB/oversight body are included in the manuscript.

Yes

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Data Availability

Data from patient records used in the study will not be available due to inability to fully anonymise up to the Information Commissioner Office (ICO) standards and ethical requirements. A subset of the dataset limited to anonymisable information is available on request to researchers with suitable training in information governance and human confidentiality protocols subject to approval by the King's College Hospital Information Governance committee and Shanghai East Hospital committee; applications for research access to KCH cohort data should be sent to [kch-tr.cogstackrequests@nhs.net] and applications for research access to Wuhan cohort data should be sent to [ting.shi{at}ed.ac.uk]. This dataset cannot be released publicly due to the risk of re-identification of such granular individual-level data. Risk prediction models are published at (https://covid.datahelps.life/prediction/).

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted May 03, 2020.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Risk prediction for poor outcome and death in hospital in-patients with COVID-19: derivation in Wuhan, China and external validation in London, UK
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Risk prediction for poor outcome and death in hospital in-patients with COVID-19: derivation in Wuhan, China and external validation in London, UK
Huayu Zhang, Ting Shi, Xiaodong Wu, Xin Zhang, Kun Wang, Daniel Bean, Richard Dobson, James T Teo, Jiaxing Sun, Pei Zhao, Chenghong Li, Kevin Dhaliwal, Honghan Wu, Qiang Li, Bruce Guthrie
medRxiv 2020.04.28.20082222; doi: https://doi.org/10.1101/2020.04.28.20082222
Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
Citation Tools
Risk prediction for poor outcome and death in hospital in-patients with COVID-19: derivation in Wuhan, China and external validation in London, UK
Huayu Zhang, Ting Shi, Xiaodong Wu, Xin Zhang, Kun Wang, Daniel Bean, Richard Dobson, James T Teo, Jiaxing Sun, Pei Zhao, Chenghong Li, Kevin Dhaliwal, Honghan Wu, Qiang Li, Bruce Guthrie
medRxiv 2020.04.28.20082222; doi: https://doi.org/10.1101/2020.04.28.20082222

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Public and Global Health
Subject Areas
All Articles
  • Addiction Medicine (179)
  • Allergy and Immunology (431)
  • Anesthesia (99)
  • Cardiovascular Medicine (940)
  • Dentistry and Oral Medicine (178)
  • Dermatology (109)
  • Emergency Medicine (260)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (418)
  • Epidemiology (8967)
  • Forensic Medicine (4)
  • Gastroenterology (417)
  • Genetic and Genomic Medicine (1938)
  • Geriatric Medicine (190)
  • Health Economics (400)
  • Health Informatics (1320)
  • Health Policy (657)
  • Health Systems and Quality Improvement (517)
  • Hematology (211)
  • HIV/AIDS (416)
  • Infectious Diseases (except HIV/AIDS) (10764)
  • Intensive Care and Critical Care Medicine (570)
  • Medical Education (199)
  • Medical Ethics (52)
  • Nephrology (221)
  • Neurology (1815)
  • Nursing (108)
  • Nutrition (271)
  • Obstetrics and Gynecology (351)
  • Occupational and Environmental Health (469)
  • Oncology (992)
  • Ophthalmology (296)
  • Orthopedics (111)
  • Otolaryngology (182)
  • Pain Medicine (126)
  • Palliative Medicine (44)
  • Pathology (265)
  • Pediatrics (576)
  • Pharmacology and Therapeutics (275)
  • Primary Care Research (234)
  • Psychiatry and Clinical Psychology (1891)
  • Public and Global Health (4107)
  • Radiology and Imaging (674)
  • Rehabilitation Medicine and Physical Therapy (361)
  • Respiratory Medicine (547)
  • Rheumatology (224)
  • Sexual and Reproductive Health (190)
  • Sports Medicine (177)
  • Surgery (206)
  • Toxicology (38)
  • Transplantation (109)
  • Urology (80)