Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

An external validation of the QCovid risk prediction algorithm for risk of mortality from COVID-19 in adults: national validation cohort study in England

View ORCID ProfileVahe Nafilyan, View ORCID ProfileBen Humberstone, Nisha Mehta, Ian Diamond, Carol Coupland, Luke Lorenzi, Piotr Pawelek, Ryan Schofield, Jasper Morgan, Paul Brown, View ORCID ProfileRonan Lyons, View ORCID ProfileAziz Sheikh, View ORCID ProfileJulia Hippisley-Cox
doi: https://doi.org/10.1101/2021.01.22.21249968
Vahe Nafilyan
aOffice for National Statistics, Cardiff Road, Newport, NP10 8XG
Roles: Principal Statistician
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Vahe Nafilyan
  • For correspondence: vahe.nafilyan@ons.gov.uk
Ben Humberstone
aOffice for National Statistics, Cardiff Road, Newport, NP10 8XG
Roles: Deputy Director
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Ben Humberstone
Nisha Mehta
bOffice of the Chief Medical Officer, Department of Health & Social Care
Roles: Clinical Adviser
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ian Diamond
aOffice for National Statistics, Cardiff Road, Newport, NP10 8XG
Roles: National Statistician
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Carol Coupland
cDivision of Primary Care, School of Medicine, University of Nottingham (CC)
Roles: Professor of Medical Statistics in Primary Care
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Luke Lorenzi
aOffice for National Statistics, Cardiff Road, Newport, NP10 8XG
Roles: Senior Data Scientist
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Piotr Pawelek
aOffice for National Statistics, Cardiff Road, Newport, NP10 8XG
Roles: Senior Statistician
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ryan Schofield
aOffice for National Statistics, Cardiff Road, Newport, NP10 8XG
Roles: Statistician
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jasper Morgan
aOffice for National Statistics, Cardiff Road, Newport, NP10 8XG
Roles: Data Manager
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Paul Brown
aOffice for National Statistics, Cardiff Road, Newport, NP10 8XG
Roles: Data Manager
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ronan Lyons
dSwansea University Medical School
Roles: Clinical Professor of Public Health
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Ronan Lyons
Aziz Sheikh
eUsher Institute, University of Edinburgh
Roles: Professor of Primary Care Research
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Aziz Sheikh
Julia Hippisley-Cox
fNuffield Department of Primary Health Care Sciences, University of Oxford, Woodstock Road, Oxford, OX2 6GG
Roles: Professor of Clinical Epidemiology & General Practice
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Julia Hippisley-Cox
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

SUMMARY

Background To externally validate a risk prediction algorithm (QCovid) to estimate mortality outcomes from COVID-19 in adults in England.

Methods Population-based cohort study using the ONS Public Health Linked Data Asset, a cohort based on the 2011 Census linked to Hospital Episode Statistics, the General Practice Extraction Service Data for pandemic planning and research, radiotherapy and systemic chemotherapy records. The primary outcome was time to COVID-19 death, defined as confirmed or suspected COVID-19 death as per death certification. Two time periods were used: (a) 24th January to 30th April 2020; and (b) 1st May to 28th July 2020. We evaluated the performance of the QCovid algorithms using measures of discrimination and calibration for each validation time period.

Findings The study comprises 34,897,648 adults aged 19-100 years resident in England. There were 26,985 COVID-19 deaths during the first time-period and 13,177 during the second. The algorithms had good calibration in the validation cohort in both time periods with close correspondence of observed and predicted risks. They explained 77.1% (95% CI: 76.9% to 77.4%) of the variation in time to death in men in the first time-period (R2); the D statistic was 3.76 (95% CI: 3.73 to 3.79); Harrell’s C was 0.935 (0.933 to 0.937). Similar results were obtained for women, and in the second time-period. In the top 5% of patients with the highest predicted risks of death, the sensitivity for identifying deaths in the first time period was 65.9% for men and 71.7% for women. People in the top 20% of predicted risks of death accounted for 90.8% of all COVID-19 deaths for men and 93.0% for women.

Interpretation The QCovid population-based risk algorithm performed well, showing very high levels of discrimination for COVID-19 deaths in men and women for both time periods. It has the potential to be dynamically updated as the pandemic evolves and therefore, has potential use in guiding national policy.

Funding National Institute of Health Research

Evidence before this study Public policy measures and clinical risk assessment relevant to COVID-19 need to be aided by rigorously developed and validated risk prediction models. A recent living systematic review of published risk prediction models for COVID-19 found most models are subject to a high risk of bias with optimistic reported performance, raising concern that these models may be unreliable when applied in practice. A population-based risk prediction model, QCovid risk prediction algorithm, has recently been developed to identify adults at high risk of serious COVID-19 outcomes, which overcome many of the limitations of previous tools.

Added value of this study Commissioned by the Chief Medical Officer for England, we validated the novel clinical risk prediction model (QCovid) to identify risks of short-term severe outcomes due to COVID-19. We used national linked datasets from general practice, death registry and hospital episode data for a population-representative sample of over 34 million adults. The risk models have excellent discrimination in men and women (Harrell’s C statistic>0.9) and are well calibrated. QCovid represents a new, evidence-based opportunity for population risk-stratification.

Implications of all the available evidence QCovid has the potential to support public health policy, from enabling shared decision making between clinicians and patients in relation to health and work risks, to targeted recruitment for clinical trials, and prioritisation of vaccination, for example.

Competing Interest Statement

JHC reports grants from National Institute for Health Research Biomedical Research Centre, Oxford, grants from John Fell Oxford University Press Research Fund, grants from Cancer Research UK (CR-UK) grant number C5255/A18085, through the Cancer Research UK Oxford Centre, grants from the Oxford Wellcome Institutional Strategic Support Fund (204826/Z/16/Z), during the conduct of the study. JHC is an unpaid director of QResearch, a not-for-profit organisation which is a partnership between the University of Oxford and EMIS Health who supply the QResearch database used for this work. JHC is a founder and shareholder of ClinRisk ltd and was its medical director until 31st May 2019. ClinRisk Ltd produces open and closed source software to implement clinical risk algorithms (outside this work) into clinical computer systems.

Funding Statement

Funding: National Institute of Health Research

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The ethics approval for the development and validation of QCovid was granted by the East Midlands-Derby Research Ethics Committee [reference 18/EM/0400].

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Data Availability

The ONS Public Health Linked Data Asset will be made available on the ONS Secure Research Service for Accredited researchers. Researchers can apply for accreditation through the Research Accreditation Service. The data will include all variables used in this analysis, except predictors based on radiotherapy and systemic chemotherapy records, which cannot be shared.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.
Back to top
PreviousNext
Posted January 25, 2021.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
An external validation of the QCovid risk prediction algorithm for risk of mortality from COVID-19 in adults: national validation cohort study in England
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
An external validation of the QCovid risk prediction algorithm for risk of mortality from COVID-19 in adults: national validation cohort study in England
Vahe Nafilyan, Ben Humberstone, Nisha Mehta, Ian Diamond, Carol Coupland, Luke Lorenzi, Piotr Pawelek, Ryan Schofield, Jasper Morgan, Paul Brown, Ronan Lyons, Aziz Sheikh, Julia Hippisley-Cox
medRxiv 2021.01.22.21249968; doi: https://doi.org/10.1101/2021.01.22.21249968
Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
Citation Tools
An external validation of the QCovid risk prediction algorithm for risk of mortality from COVID-19 in adults: national validation cohort study in England
Vahe Nafilyan, Ben Humberstone, Nisha Mehta, Ian Diamond, Carol Coupland, Luke Lorenzi, Piotr Pawelek, Ryan Schofield, Jasper Morgan, Paul Brown, Ronan Lyons, Aziz Sheikh, Julia Hippisley-Cox
medRxiv 2021.01.22.21249968; doi: https://doi.org/10.1101/2021.01.22.21249968

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Public and Global Health
Subject Areas
All Articles
  • Addiction Medicine (161)
  • Allergy and Immunology (414)
  • Anesthesia (90)
  • Cardiovascular Medicine (857)
  • Dentistry and Oral Medicine (159)
  • Dermatology (97)
  • Emergency Medicine (248)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (393)
  • Epidemiology (8557)
  • Forensic Medicine (4)
  • Gastroenterology (383)
  • Genetic and Genomic Medicine (1749)
  • Geriatric Medicine (167)
  • Health Economics (372)
  • Health Informatics (1239)
  • Health Policy (620)
  • Health Systems and Quality Improvement (467)
  • Hematology (196)
  • HIV/AIDS (372)
  • Infectious Diseases (except HIV/AIDS) (10292)
  • Intensive Care and Critical Care Medicine (553)
  • Medical Education (192)
  • Medical Ethics (51)
  • Nephrology (211)
  • Neurology (1676)
  • Nursing (97)
  • Nutrition (249)
  • Obstetrics and Gynecology (326)
  • Occupational and Environmental Health (450)
  • Oncology (928)
  • Ophthalmology (263)
  • Orthopedics (101)
  • Otolaryngology (172)
  • Pain Medicine (112)
  • Palliative Medicine (40)
  • Pathology (252)
  • Pediatrics (534)
  • Pharmacology and Therapeutics (248)
  • Primary Care Research (207)
  • Psychiatry and Clinical Psychology (1765)
  • Public and Global Health (3835)
  • Radiology and Imaging (623)
  • Rehabilitation Medicine and Physical Therapy (320)
  • Respiratory Medicine (520)
  • Rheumatology (208)
  • Sexual and Reproductive Health (166)
  • Sports Medicine (158)
  • Surgery (190)
  • Toxicology (36)
  • Transplantation (101)
  • Urology (76)