Summary
Clinical and genetic risk factors for severe COVID-19 are often considered independently and without knowledge of the magnitudes of their effects on risk. Using SARS-CoV-2 positive participants from the UK Biobank, we developed and validated a clinical and genetic model to predict risk of severe COVID-19. We used multivariable logistic regression on a 70% training dataset and used the remaining 30% for validation. We also validated a previously published prototype model. In the validation dataset, our new model was associated with severe COVID-19 (odds ratio per quintile of risk=1.77, 95% confidence interval [CI]=1.64, 1.90) and had excellent discrimination (area under the receiver operating characteristic curve=0.732, 95% CI=0.708, 0.756). We assessed calibration using logistic regression of the log odds of the risk score, and the new model showed no evidence of over- or under-estimation of risk (α=−0.08; 95% CI=−0.21, 0.05) and no evidence or over- or under-dispersion of risk (β=0.90, 95% CI=0.80, 1.00). Accurate prediction of individual risk is possible and will be important in regions where vaccines are not widely available or where people refuse or are disqualified from vaccination, especially given uncertainty about the extent of infection transmission among vaccinated people and the emergence of SARS-CoV-2 variants of concern.
Key results
Accurate prediction of the risk of severe COVID-19 can inform public heath interventions and empower individuals to make informed choices about their day-to-day activities.
Age and sex alone do not accurately predict risk of severe COVID-19.
Our clinical and genetic model to predict risk of severe COVID-19 performs extremely well in terms of discrimination and calibration.
Introduction
The COVID-19 pandemic continues to dominate global public health, with countries having varying success with infection control measures and social distancing protocols [1], Coupled with this are the logistical challenges with the distribution of vaccines [2] and the emergence of SARS-CoV-2 variants of concern [3, 4]. Of those who become infected with SARS-CoV-2, 10–15% will develop severe COVID-19 requiring hospitalisation and 5% will require intensive care [5]. At all stages of the pandemic, there has been an urgent need for accurate quantification of risk of severe COVID-19 to inform protection from infection for those at increased risk.
Epidemiological analyses have recognized that sex and increasing age are risk factors for severe COVID-19 and that common medical comorbidities contribute to individual risk [6-8]. Our previous analysis showed that the effects of sex and age are attenuated when comorbidities are taken into account [9]. The effect of human genetic variation on COVID-19 severity has been examined by the COVID-19 Host Genetics Initiative, which has now released several meta-analyses of the available genome-wide association studies of COVID-19 severity [10, 11]. Using population controls, Ellinghaus et al. [12] identified two loci (3p21.31 and 9q34.2) as being strongly associated with respiratory failure from COVID-19 and Shelton et al. [13] identified the 3p21.31 locus as being associated with severe COVID-19. Also using population controls, Pairo-Castineira et al. [14] identified eight single-nucleotide polymorphisms (SNPs) that achieved genome-wide significance for intensive care admission and identified six SNPs (two of which were also in the panel of eight SNPs) associated with risk of hospitalization.
The emergency authorization of SARS-CoV-2 vaccines [15] does not diminish the value of accurate prediction of individual risk of severe COVID-19. Extensive vaccine disqualification criteria (such as pre-existing conditions, pregnancy, and age), vaccine hesitancy, uncertainty as to whether the vaccines are effective against emerging variants of concern [4], and an unknown extent to which vaccines prevent the transmission of infection mean that many people will be at risk of severe COVID-19 should they become infected with SARS-CoV-2.
We previously developed a prototype risk model [9] based upon early data from the UK Biobank [16, 17] and SNPs identified from the COVID-19 Host Genetics Initiative Release 2 meta-analysis of hospitalized vs non-hospitalized COVID-19 cases (which was at that time almost exclusively UK Biobank samples) [10, 18]. Our prototype model appeared to perform well but was based on a small sample size from the first wave of the pandemic [9].
We decided not to attempt validation in this dataset because of our concern about the representativeness of the data (the SARS-CoV-2 testing data was ascertained early in the pandemic when the limitations on testing availability in the United Kingdom meant that mild and asymptomatic cases were not identified).
In the interim, the UK Biobank has released further data from participants confirmed to be infected with SARS-CoV-2. This latest data release (2205 cases and 5416 controls) has a larger proportion of non-hospitalized people, providing more confidence that they are a more representative. In this paper, we perform a validation study of our prototype model and to develop and validate a new clinical and genetic model to predict risk of severe COVID-19.
Methods
UK Biobank data and eligibility
Since our first paper on the development of a risk prediction model for severe COVID-19 [9], the UK Biobank [16, 17] has accumulated a large number of additional SARS-CoV-2 test results [19]. For this analysis, we downloaded an updated results file on 8 January 2021. As in our first paper, eligible participants were active UK Biobank participants with a positive SARS-CoV-2 test result and who had SNP and hospital data available [9]. Of the 47990 UK Biobank participants with at least one SARS-CoV-2 test result, 8672 (18.1%) had a positive test result, and of these, 7621 met our eligibility criteria.
As we did previously [9], we used source of test result as a proxy for severity of disease, where inpatient results were considered severe disease (cases) and outpatient results were considered non-severe disease (controls). If a participant had more than one test result, we classified them as having severe disease if at least one of their results was from an inpatient setting. Of the 7621 eligible participants, 2205 (28.9%) were cases and 5416 (71.7%) were controls.
Data extraction
We used UK Biobank clinical and genetic data that we had previously downloaded (see Table 1). We used Plink version 1.9 [20, 21] to extract SNP data from the UK Biobank imputation dataset. We extracted genotypes of the 64 SNPs that were used to calculate the SNP score in our prototype model [9] and the 12 SNPs from Pairo-Castineira et al. [14] We also identified 43 SNPs from the B1_ALL (hospitalized versus non-hospitalized cases of COVID-19) results of the COVID19-hg GWAS meta-analyses round 4, conducted by the COVID-19 Host Genetics Initiative consortium.[10, 22] These SNPs were selected by pruning variants with a P value of greater than 10−5 and linkage disequilibrium variants that had an R2 of greater than 0.5 for all populations. Of these 43 SNPS, 40 were available for extraction in the UK Biobank imputation dataset. The SNPs considered in the current paper are listed in Supplementary Table S1.
Validation of prototype model
For the validation of our prototype risk model [9], we used the 1234 cases and 4805 controls that were not included in our previous paper. We constructed relative risk scores for both the clinical model and the combined clinical and SNP score model using the exponent of the sum of the intercept and the beta coefficients for each risk factor in the prototype model [9].
Development and validation of the new model
To develop a new model to predict risk of severe COVID-19, we used all of the available data and randomly divided it into a 70% training dataset and a 30% validation dataset (ensuring that the datasets were balanced for case and control status). We used multiple imputation with 20 imputations to address the missing data for body mass index (linear regression) and the SNP data (predictive mean matching) for the development of the new model in the training dataset. To more closely reflect the availability of data in the real world, we did not use imputed data in the validation dataset.
The clinical variables considered for inclusion in the new model were age, sex, body mass index (BMI), ethnicity (Caucasian vs other), ABO blood type and the following chronic health conditions: asthma, autoimmune disease (rheumatoid arthritis, lupus or psoriasis), haematological cancer, non-haematological cancer, cerebrovascular disease, diabetes, heart disease, hypertension, immunocompromised, kidney disease, liver disease and respiratory disease (excluding asthma). Dummy variables were used for the categorical classifications of age and ABO blood type.
The SNPs selected for consideration in the development of the new model came from three sources: (i) the 64 SNPs from our prototype model [9], (ii) the 12 SNPs from the paper by Pairo-Castineira et al. [14], and (iii) the 40 SNPs newly selected from the results of the COVID-19 Host Genetics Initiative’s COVID19-hg GWAS meta-analyses round 4 meta-analysis of non-hospitalized versus hospitalized cases of COVID-19 [10, 22]. To avoid reliance on potentially inaccurate summary statistics to construct a polygenic risk score, we used unadjusted logistic regression in the multiple imputation training dataset to identify the subset of SNPS that were associated with risk of severe COVID-19 with P<0.05 (see Supplementary Table S1) and used these as individual risk factors (with a per allele effect) to build our new model.
Statistical methods
Development of new model
We used multivariable logistic regression in the multiple imputation training dataset to develop the new model to predict risk of severe COVID-19. We began with a model that included all of the clinical variables and the SNPs with unadjusted associations with severe COVID-19. We then used backwards stepwise selection to develop the most parsimonious model. For the removed variables we made a final determination on their inclusion or exclusion by adding them one at a time to the parsimonious model. To directly compare the effect sizes of the variables in the final model, regardless of the scale on which they were measured, we used the odds per adjusted standard deviation [23]. We used the intercept and beta coefficients from the new model to calculate the COVID-19 risk score (as a % risk) for all eligible UK Biobank participants.
Model performance
The association between risk score and severe COVID-19 was assessed using logistic regression to estimate the OR per quintile of risk score. We assessed model discrimination using the area under the receiver operating characteristic curve (AUC). Where warranted, we plotted the receiver operating characteristic curve of the model.
We assessed calibration using logistic regression of the log odds of the risk score to estimate the intercept and the slope (beta coefficient). An intercept close to 0 indicates good calibration, while an intercept of less than 0 indicates overall overestimation and an intercept of greater than 0 indicates overall underestimation of risk.
In terms of the dispersion of the risk score, a slope of close to 1 indicates good estimation across the spectrum of risk. A slope of less than 1 means that the predicted probabilities do not vary enough (i.e. underestimation of true high risk and overestimation of true low risk). Conversely, a slope of greater than one means that the predicted probabilities vary too much (i.e. underestimation of true low risk and overestimation of true high risk). Where helpful, we also used a calibration plot to illustrate the fit of a model.
We used Stata (version 16.1) [24] for analyses; all statistical tests were two-sided and P<0.05 was considered nominally statistically significant.
Ethics approval
The UK Biobank has Research Tissue Bank approval (REC #11/NW/0382) that covers analysis of data by approved researchers. All participants provided written informed consent to the UK Biobank before data collection began. This research has been conducted using the UK Biobank resource under Application Number 47401.
Data availability statement
The data underlying this article was provided by the UK Biobank and we do not have permission to share the data. Researchers wishing to access the data used in this study can apply directly to the UK Biobank at https://www.ukbiobank.ac.uk/register-apply/. Stata 16.1 code for the analysis is available from the corresponding author on request.
Results
In the results file downloaded on 8 January 2021, there were 2205 eligible cases with severe COVID-19 and 5416 eligible controls with non-severe COVID-19.
Validation of prototype model
Characteristics of the new UKB participants (1234 cases and 4805 controls) with positive SARS-CoV-2 test results are shown in Supplementary Table S2.
The odds ratio (OR) per quintile showed that the clinical risk score was strongly associated with severe COVID-19 (OR=1.70; 95% confidence interval [CI]=1.62, 1.79; P<0.001) and that the combined clinical and SNP risk score was less strongly associated with severe COVID-19 (OR=1.45; 95% CI=1.38, 1.52; P<0.001); there was no association with severe COVID-19 for the SNP score (OR=0.98; 95% CI=0.94, 1.03; P=0.5). The discrimination of cases and controls was excellent for the clinical score (AUC=0.711; 95% CI=0.694, 0.727), lower for the combined clinical and SNP score (AUC=0.657; 95% CI=0.639, 0.674) and poor for the SNP score alone (AUC=0.491; 95% CI=0.473, 0.509).
Assessment of model calibration showed that overall, risk was overestimated for both the clinical risk model (α=−1.72; 95% CI=−1.80, −1.65; P<0.001) and the clinical and SNP model (α=−1.63; 95% CI=−1.71, −1.54; P<0.001). For the clinical model, there was no evidence of poor dispersion (β=1.03, 95% CI=0.94, 1.12, P=0.5), while the predictions of the combined clinical and SNP model varied too much (β=0.59, 95% CI=0.52, 0.65, P<0.001).
Development and validation of the new model
Table 1 shows the characteristics of the 1544 cases and 3791 controls in the 70% training dataset and the 661 cases and 1,625 controls in the 30% validation data set. In the training dataset, the mean age was 69.8 years (SD=8.6) for cases and 64.6 years (SD=8.4) for controls, and the mean BMI was 29.3 kg/m2 (SD=5.3) for cases and 28.0 kg/m2 (SD=4.9) for controls. In the validation dataset, the mean age was 69.7 years (SD=8.7) for cases and 64.4 years (SD=8.4) for controls, and the mean BMI was 29.4 kg/m2 (SD=5.6) for cases and 28.3 kg/m2 (SD=5.0) for controls.
Training
In the age and sex model, being male and being in one of the four older age groups conferred a substantially increased risk of severe COVID-19 (Table 2), with an OR=1.60 for being male and ORs ranging from 2.74 for the age groups from 65–69 years to 4.95 for the 80+ years group. Direct comparison of the effect size of each variable showed that the age group 75–79 years was the strongest risk factor (with odds per adjusted standard deviation of 1.58), followed by the 70–74 and 80–84 groups (with odds per adjusted standard deviations of 1.42 and 1.34, respectively).
The new model was developed from the variables in Table 1, which include the clinical variables and the 14 SNPs identified as having unadjusted per allele ORs with P-values <0.05 (see Supplementary Table S1). The variables retained in the new model are shown in Table 3 and comprise three age groups (70–74, 75–79 and 80–84 years), sex, ethnicity, body mass index, six comorbidities and seven SNPs. Compared with the age and sex model, the effects of sex and age group were attenuated in the new model, with an OR=1.27 for being male, the 70–74 years age group not being at increased risk, and ORs for the other age groups ranging from 1.77 for the 70–74 years group to 2.76 for the 80+ years group.
Direct comparison of the effect size of each variable showed that respiratory disease was the strongest risk factor with odds per adjusted standard deviation of 1.35, followed by the three older age groups with odds per adjusted standard deviations of 1.20 to 1.29). The other clinical risk factors and SNPs had odds per adjusted standard deviation in the range 1.07 to 1.13 (or the equivalent protective effect).
The age and sex model had good discrimination of cases and controls with an AUC of 0.676 (95% CI=0.659, 0.692) but the new model with an AUC of 0.752 (95% CI=0.737, 0.767) was a substantial improvement (χ2=149.40, df=1, P<0.001).
Validation
In the non-imputed validation dataset, the age and sex model and the new model were associated with severe COVID-19. The OR per quintile for the age and sex model was 1.49 (95% CI=1.40, 1.59; P<0.001), while the new model had a substantially higher OR per quintile of 1.77 (95% CI=1.64, 1.90; P<0.001).
In terms of discrimination between cases and controls, the age and sex model had an AUC of 0.671 (95% CI=0.646, 0.696), while the new model with an AUC of 0.732 (95% CI=0.708, 0.756) was a substantial improvement (χ2=41.23, df=1, P<0.001). The receiver operating characteristic curves for both models are shown in Figure 1.
Both models were well calibrated with no evidence of overall overestimation or underestimation for the age and sex model (α=−0.02; 95% CI=−0.18, 0.13; P=0.7) or the new model (α=−0.08; 95% CI=−0.21, 0.05; P=0.3). There was also no evidence of under or over dispersion for the age and sex model (β=0.96, 95% CI=0.81, 1.10, P=0.6) and for the new model (β=0.90, 95% CI=0.80, 1.00, P=0.06). Calibration plots for both models are shown in Figure 2.
Probability of severe COVID-19 in whole UK Biobank
We calculated the probability of severe COVID-19 for all UK Biobank participants who met our eligibility criteria for this study; the distributions are shown in Figure 3, and the distribution of the new model by 5-year age group are shown in Supplementary Figure S1. Using the age and sex model, the mean probability was 0.32 (SD=0.13) and ranged from a minimum of 0.15 to a maximum of 0.56. Using the new model, the mean probability was 0.27 (SD=0.16) and the range was from 0.04 to 0.98, a much wider range than for the age and sex model.
Discussion
An accurate test to predict risk of severe COVID-19 can inform prioritization of vaccine doses to those most at risk [25] and will be useful in regions in which vaccination is not widespread enough to provide herd immunity (either through unavailability or vaccine hesitancy), if available vaccines are not effective against variants of SARS-CoV-2, or if available vaccines are not indicated for some people. On an individual level, knowledge of personal risk can empower people to make informed choices about their day-to-day activities, including targeted social distancing in the workplace [26] or other crowded places.
The validation of the clinical component of our prototype model confirmed that it performed well with good discrimination (AUC=0.711), but overall, it overestimated risk. The SNP score component of the prototype model was not confirmed in the validation dataset and is likely due to the prototype model having been developed in dataset with a high prevalence of severe COVID-19.
Given the failure to confirm our prototype SNP-based risk score, we incorporated SNPs in the new model without relying on published summary statistics and without assumptions as to the identity of the risk allele. We included the SNPs as individual risk factors and estimated the per allele OR for each. By doing so, we were able to identify the subset of SNPs and clinical risk factors that were informative for predicting risk. These risk factors are all important to risk prediction, and characterization of the SNP genotypes is as important as ascertaining clinical information.
From our initial list of 116 SNPs (Supplementary Table S1), we considered 14 for inclusion in our model and retained seven, none of which were in the 3p21.31 locus identified by others [12-14, 22]. Functionally, most of the SNPs retained in our new model are associated with genes that play a role in infection pathways or immunity. The immune function and chromatin remodelling family of GATA transcription factors are associated by the inclusion of SNPs near HIVEP1 (rs10755709), which encodes a viral-infection regulation transcription factor, and GATA3 (rs71481792) [27, 28]. ALPK1 and TIFA are closely downstream of rs112641600 and both have adaptive and innate signal transduction roles and pro-inflammatory functions [29]. MSR1, upstream of rs118072448, is a macrophage scavenger receptor and implicated in a broad range of disease types including host viral defence [30] and PSAT1 is associated with glutamine metabolic reprogramming by SARS-CoV-2 and viral mRNA translation [31].
In the development of the new model, the strongest risk factor was respiratory disease (with an odds per standard deviation of 1.35; Table 3). The older age groups (70–74, 75–79, and 80+ years) and being male all had odds per standard deviations of 1.20 to 1.29. The other risk factors (the seven SPNs, ethnicity, body mass index, cancer history (haematological and non-haematological), cerebrovascular disease, diabetes, hypertension, and kidney disease) all had odds per adjusted standard deviations in the range 1.07 to 1.13 (or the equivalent protective effect).
In the non-imputed validation dataset, the new model performed very well with an AUC of 0.732 (compared with an AUC of 0.752 in the training dataset). Importantly, the new model was well calibrated, showing no evidence of problems with the overall estimation of risk or the dispersion of risk predictions. The validation of the new model also illustrates the importance of considering risk factors beyond age and sex in predicting risk of severe COVID-19. The new model was a substantial improvement over the age and sex model, in terms of the OR per quintile (OR=1.77 and OR=1.49, respectively) and the discrimination of cases and controls (AUC=0.732 and AUC=0.671, respectively). The new model also allows stratification across a wide range of risk (Figure 3B) so that, for example, a healthy person aged 75 years might have a lower risk of severe COVID-19 than a 50-year-old person with several risk factors.
A limitation of this study is that, through necessity, we used hospitalization as a proxy for COVID-19 severity and the outcome measure may have been misclassified for some participants. This would have attenuated the observed associations and it is possible that some risk factors have been omitted unnecessarily. Nevertheless, we are confident in the variables retained. We were also unable to develop models for other important endpoints such as intensive care admission or death.
The progression of the COVID-19 pandemic has seen people experience chronic symptoms, and some of these people will have had only a mild original infection [5]. Identifying people who are at increased risk of chronic disease is an obvious direction for future research. Another direction for future research is to investigate whether our model for the prediction of severe COVID-19 is applicable for the new SARS-CoV-2 variants of concern, which have been reported to have increased transmissibility, virulence and antigenicity and cause more severe disease [3, 4]. Further validation of our new model is required in independent datasets, especially those in which the SARS-CoV-2 variant has been characterized.
Clear benefits of our new model for predicting risk of severe COVID-19 are that the required clinical data is simple to collect and that the genetic information is amenable to high-throughput genotyping, with rapid turnaround that is essential for the present pandemic. In the light of the uncertainty of the future of the COVID-19 pandemic, accurate knowledge of individual risk of severe COVID-19 can make an important contribution to healthcare on both a population and a personal level.
Data Availability
We used data from the UK Biobank (under Application Number 47401) for these analyses and do not have permission to share the data. Researchers wishing to access the data used in this study can apply directly to the UK Biobank at https://www.ukbiobank.ac.uk/register-apply/.
Financial support
This study was fully funded by Genetic Technologies Limited.
Conflict of interest
GSD, NMM, and RA are employees of Genetic Technologies Limited. Genetic Technologies Limited had no role in the conceptualization, design, data analysis, decision to publish or preparation of the manuscript.
Aspects of this manuscript are covered by Provisional Patent Application AU_2021900392 (pending), Methods of assessing risk of developing a severe response to Coronavirus infection. GSD, NMM, and RA are named inventors on the patent application, which is assigned to Genetic Technologies Limited.
Supplementary Information
Supplementary figure
Supplementary tables
Acknowledgements
We wish to thank Mr Lawrence Whiting for his invaluable expertise in the management of large data files from the UK Biobank.
Footnotes
We have removed the validation analyses of the SNPs identified by Pairo-Castineira et al (Nature 2021; 591: 92-98) and have written these up as a separate analysis that we will also post on medRxiv.