ABSTRACT
Objectives COVID-19 pandemic poses a burden on hospital resources and intensive care unit (ICU) occupation. This study aimed to provide a scoring system that, assessed upon first-contact evaluation at the emergency department, predicts the need for ICU admission.
Methods We prospectively assessed patients admitted to a COVID-19 reference center in Mexico City between March 16th and May 21st, and split them into development and validation cohorts. Patients were segregated into a group that required admission to ICU, and a group that never required ICU admission and was discharged from hospitalization. By logistic regression, we constructed predictive models for ICU admission, including clinical, laboratory, and imaging findings from the emergency department evaluation. The ABC-GOALS score was created by assigning values to the weighted odd ratios. The score was compared to other COVID-19 and pneumonia scores through the area under the curve (AUC).
Results We included 569 patients divided into development (n=329) and validation (n=240) cohorts. One-hundred-fifteen patients from each cohort required admission to ICU. The clinical model (ABC-GOALSc) included sex, obesity, the Charlson comorbidity index, dyspnea, arterial pressure, and respiratory rate at triage evaluation. The clinical plus laboratory model (ABC-GOALScl) added serum albumin, glucose, lactate dehydrogenase, and S/F ratio to the clinical model. The model that included imaging (ABC-GOALSclx) added the CT scan finding of >50% lung involvement. The model AUC were 0.79 (95%CI 0.74-0.83) and 0.77 (95%CI 0.71-0.83), 0.86 (95%CI 0.82-0.90) and 0.87 (95%CI 0.83-0.92), 0.88 (95%CI 0.84-0.92) and 0.86 (95%CI 0.81-0.90) for the clinical, laboratory and imaging models in the development and validation cohorts, respectively. The ABC-GOALScl and ABC-GOALSclx scores outperformed other COVID-19 and pneumonia-specific scores.
Conclusion The ABC-GOALS score is a tool to evaluate patients with COVID-19 at admission to the emergency department, which allows to timely predict their risk of admission to an ICU.
INTRODUCTION
The presence of a new severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was first reported in China and has subsequently spread to all regions of the world, straining the health systems of many countries1. Viral pneumonia associated with SARS-CoV-2 has been officially denominated as coronavirus disease 2019 (COVID-19)2.
Approximately 5 to 33% of patients with COVID-19 pneumonia will be admitted to an intensive care unit (ICU)3–5. Previous studies have identified several risk factors associated with a severe course of COVID-19 pneumonia and its progression to acute respiratory distress syndrome 6,7. These risk factors may be categorized into patient characteristics obtained through the medical interview (e.g. age, comorbidities, symptoms)6–11; vital signs obtained from triage evaluation (e.g. respiratory rate, arterial pressure)12,13; laboratory abnormalities including inflammatory, coagulation and organ-specific studies (e.g. lactate dehydrogenase, D-dimer, fibrinogen, cardiac troponins, liver function tests, among others) 6,7,9–11,14; and lung imaging findings (e.g. number of affected lobes, estimated pneumonia extension) 15,16.
In several countries, including Mexico, hospital reconversion and temporary care centers have been implemented to cope with a large number of COVID-19 patients 17 However, many of these centers are prepared to care for patients with supplemental oxygen requirements in general wards, but just a few patients who require admission to an intensive care unit (ICU) and mechanical ventilation.
The study aimed to derive and validate simple risk prediction models to anticipate the need for admission to an ICU. These models include clinical, laboratory, and image findings obtained at the first-contact evaluation to aid triage, decision-making, and timely referral to maintain healthcare system capacity.
METHODS
This is a prospective observational cohort study. All consecutive adult (>18 years) patients hospitalized at Instituto Nacional de Ciencias Médicas y Nutrición Salvador Zubirán, a referral center for COVID-19 patients at Mexico City, were evaluated for this analysis. The study was approved by the local Research and Ethics Board (CAI-3368-20-20-1) that waived the use of a written informed consent form due to the study’s nature.
All patients hospitalized between March 16th and April 30th were assigned to the development cohort, and patients hospitalized between May 1st and May 21th were assigned to the validation cohort. The study population was segregated into two groups: 1) patients who required admission to an ICU at any time during their hospitalization, and 2) patients admitted to hospital general wards who were discharged from hospitalization without ever been considered for admission to an ICU. Patients were censored for each group once they were admitted to the ICU or by the date of discharge from hospitalization to home, respectively. All patients who remained hospitalized in general wards by the end of the study were not included in this analysis as it was considered that their risk for ICU admission was still active.
All patients had COVID-19 pneumonia diagnosed by chest computed tomography (CT), and SARS-CoV-2 infection confirmed in respiratory specimens using real-time reverse-transcriptase polymerase chain reaction (RT-PCR) assay by local testing and confirmed at a central laboratory. Patients were considered positive if the initial test results were positive, or if it was negative but repeat testing was positive.
The following variables were obtained by the time of the triage and emergency department (ED) evaluation: demographic variables, previous medical and medication history including smoking; symptoms, physical examination including weight, height, and vital signs; laboratory evaluation including arterial blood gas analysis, inflammatory biomarkers, troponin-I levels, complete blood count and blood chemistries; and chest CT-scan findings. All patients underwent chest CT-scans that were evaluated by experienced specialists; lung involvement was semi-quantitatively classified as mild (<20%), moderate (20-50%), or severe (>50%).
For each patient, we calculated the Charlson comorbidity index (an index that predicts 10-year survival in patients with multiple comorbidities) 18, National Early Warning Score (NEWS) 2 19, Sequential Organ Failure Assessment (SOFA) score 20, CURB-65 score for pneumonia severity 21, the MuLBSTA score for viral pneumonia mortality 22, the ROX index to predict the risk of intubation 23, the CALL score model for prediction of progression risk in COVID-19 24 and the COVID-GRAM score for the risk of critical illness in COVID-1925. All scores were calculated at admission to the ED. Although some of these scores were not derived to evaluate the risk of admission to an ICU, we evaluated their value if repurposed for this end.
The need for admission to critical care was determined by the medical team in charge of the patient and included the need for mechanical ventilation or high-dose vasopressors. Patients hospitalized in medical wards were treated with supplementary oxygen (nasal cannula or non-rebreathing oxygen mask) but were never considered for intubation and ICU admission. All patients in the latter group were censored once discharged from hospitalization to home after improvement. Clinical outcomes for patients admitted to ICU were monitored up to May 25th’ 2020.
Statistical analysis
The distribution of continuous variables was evaluated by the Kolmogorov-Smirnov test. Variables are described as number (relative frequency) or median (interquartile range [IQR]) as appropriate. Characteristics at admission between study groups were compared by the Mann-Whitney U test. There were less than 2% missing values for all variables. In the case of missing data, variables were imputed by using multiple imputations 26
For score derivation, all variables were evaluated by bivariate logistic regression analysis. All variables with p-values <0.05 were considered for the multivariate logistic regression analysis. The best logistic regression analysis model was constructed by the forward stepwise selection method using maximum likelihood estimation and r-square values. Three models were constructed: a first model including only clinical variables (ABC-GOALSc), a second model that included clinical and laboratory variables (ABC-GOALScl), and a third model that included clinical, laboratory, and x-ray findings (ABC-GOALSclx). The predictive performance of each model was measured by the concordance index (C-index) and internal calibration was evaluated by 1000 bootstrap samples. The goodness of fit was evaluated by the Hosmer-Lemeshow test. To create the final scores, points were assigned by the weighted odd-ratios and approached to the closest integer for each model.
We prospectively validated the ABC-GOALS scores in an independent validation cohort. The performance of the derived scores as well as for all other scores determined at admission to predict the risk of hospitalization into an ICU were assessed by receiver operating characteristic (ROC) curves and their 95% confidence intervals.
All analyses were performed with SPSS version 24.0 (IBM, Armonk, NY, USA) and GraphPad Prism 6.0 (GraphPad Software, San Diego, CA, USA).
RESULTS
Patient characteristics at admission
A total of 705 patients with COVID-19 pneumonia and SARS-CoV-2 positive test were hospitalized during the study period. Eighty-three patients who remained hospitalized in general wards, 28 patients referred to a convalescence center, 22 early referred to other institutions, and three patients who were discharged by discharge against medical advice were excluded from the analysis. The development cohort included 329 patients and the validation cohort 240 patients (Figure 1).
Characteristics of the development cohort
The development cohort comprised 115 patients that required ICU admission and 214 patients that had been hospitalized and discharged from general wards. The median age of the derivation cohort was 49 years (IQR 41-60), 211 (64%) subjects were male, with a median Charlson comorbidity index of 1 point (IQR 0-2). There were 100 (30%) subjects with no previous comorbidity. The median days from the start of symptoms to the evaluation at the ER were 7 days (IQR 5-10). All patients received supplementary oxygen. For the group admitted to ICU, the median time from ER admission to ICU admission was 2 days (IQR 0-3 days, range 0 to 12). The median length of hospitalization for the group hospitalized in general wards was 6 days (IQR 4-9 days, range 1 to 20). Several differences in the admission variables were observed between patients who required critical care and those hospitalized in general wards and are described in Table 1 and Supplementary Table 1.
Factors associated with the need for admission to a critical care unit
An extended table with all the results obtained from the bivariate logistic regression analysis is shown in Supplementary Table 2. The best predictive models derived from multivariable logistic regression analysis are shown in Table 2. The clinical model included: male sex, the Charlson comorbidity index, obesity (BMI≥30kg/m2, not included in the Charlson index), referred dyspnea, respiratory rate, and systolic arterial pressure at the triage or ER evaluation. The clinical plus laboratory model included the same clinical variables plus serum albumin <3.5g/dL, lactate dehydrogenase above the upper limit of normal, and the hemoglobin oxygen saturation to the fraction of inspired oxygen ratio <300 (S/F ratio). The clinical plus laboratory model included all previous variables except for referred dyspnea and serum albumin and added reported lung involvement >50% in the lung CT scan. The c-statistics for clinical, clinical plus laboratory and clinical plus laboratory plus x-ray models were 0.79, 0.87, and 0.88, respectively.
Derivation of the ABC-GOALS scores
A predictive point score was constructed based on the weighted OR’s from each logistic regression model. The derived scores were defined as the ABC-GOALS (Arterial pressure, Breathlessness, Charlson, Glucose, Obesity, Albumin, LDH and S/F ratio) and were labeled as clinical (ABC-GOALSc), clinical plus laboratory (ABC-GOALScl) and clinical plus laboratory plus x-ray (ABC-GOALSclx) scores (Figure 2).
All three ABC-GOALS score variations demonstrated good accuracy in estimating the risk of admission to an ICU, with an area under the curve of 0.79 (95%CI 0.74-0.83), 0.86 (95%CI 0.82-0.90) and 0.88 (95%CI 0.84-0.92) for the ABC-GOALSc, ABC-GOALScl and ABC-GOALSclx scores, respectively (Figure 3). Calibration plots showed good agreement between the estimated and observed scores (Supplementary Figure 1).
By segregating the scores into three levels of risk of admission to ICU, the ABC-GOALSc classified patients into low- (0-2 points, mean risk 8%, 95%CI 7-9%), moderate- (3-7 points, mean risk 32%, 95%CI 30-34%) and high-risk (≥8 points, mean risk 73%, 95%CI 70-75%) of admission ICU. The respective groups for the ABC-GOALScl were low- (0-3 points, mean risk 4%, 95%CI 3-4%), moderate- (4-9 points, mean risk 30%, 95%CI 27-32%) and high-risk (≥10 points, mean risk 80%, 95%CI 78-82%) for ICU admission. Finally, the groups for the ABC-GOALSclx were low- (0-4 points, mean risk 6%, 95%CI 5-6%), moderate- (5-9 points, mean 32%, 95%CI 29-34%) and high-risk (≥10 points, mean 80%, 95%CI 78-82%) for ICU admission (Figure 2). The sensitivity and specificity for selected cutoffs of these scores are shown in Table 3.
The ABC-GOALS variants outperformed other scores that were repurposed for the prediction of admission to ICU, based on the area under the curve values (Table 3). As previously stated, most of these scores were not created to predict this outcome. For practicality, the three ABC-GOALS scores were implemented into an application for mobile devices.
Validation of the ABC-GOALS scores
The ABC-GOALS scores were prospectively validated in consecutive patients included into the validation cohort. The characteristics of the patients included in this cohort are shown in Supplementary Table 3. The area under the curve for each score was 0.77 (95%CI 0.71-0.83), 0.87 (95%CI 0.83-0.91), and 0.86 (95%CI 0.81-0.90) for the ABC-GOALSc, ABC-GOALScl, and ABC-GOALSclx, respectively (Figure 3). The ABC-GOALS scores’ predictive performance was also superior to other scores in the validation cohort (Table 3).
Outcomes
By the end of this study (May 24th), all patients hospitalized in general wards in the development and validation cohorts were discharged to home after improvement. Three patients (0.8%) were readmitted and discharged again without admission to ICU during the study period. From all patients admitted to the ICU in the development cohort (n=115), 63 (55%) have died, 37 (32%) have been discharged to their homes after improvement, and 15 (13%) remain hospitalized.
DISCUSSION
In this study, we derived and validated a prognostic score that, evaluated upon patient admission to the ED, helps predict the probability of admission to ICU during patient hospitalization. This tool may prove useful for patient assessment at first-contact evaluation and to timely refer patients to other units in case of a lack of or overcrowding of local ICUs. The ABC-GOALS scores predicted the need for admission to an ICU with greater accuracy than other reported COVID-19 prognostic scores, as well as other scores that were created for the prediction of other outcomes in sepsis or pneumonia, such as mortality.
To date, more than 4 million SARS-CoV-2 infections have been reported worldwide with more than 300,000 total deaths27. The fast spread of the disease 28 and the large number of patients admitted to hospitals has strained and overwhelmed local health systems. Among hospitalized patients, 5 to 33% will require admission to an ICU and 75% to 100% of them will require support with mechanical ventilation 3–5,8. An effective evaluation of patients with severe disease may be critical to maintain the healthcare system functional for as long as possible.
The ABC-GOALS scores summarize many of the previously reported factors that have been shown to independently associate with severe COVID-19 disease. We found that age and comorbidities predictive performance is enhanced when integrated into the Charlson comorbidity index. It has been reported in other studies that there may be interactions between age and some comorbidities such as diabetes29. Also, the Charlson comorbidity index offers the advantage of integrating several comorbidities into one score that has been previously associated with survival 18 Among clinical symptoms, dyspnea or shortness of breath is the most consistently reported symptom associated with COVID-19 severity and mortality 6,9,12,30 and was independently associated with ICU admission in this study. Obesity is a long-recognized factor for severe pulmonary infections 31 and has been consistently associated with adverse outcomes in COVID-19 32 Low systolic arterial pressure and respiratory rate at triage evaluation complete the clinical ABC-GOALS.
Among the three derived scores in this study, the clinical ABC-GOALS has an inferior performance and probably equivalent to other early warning scores. It was designed to provide a tool for areas without timely access to laboratory results. However, the clinical score remains to be externally validated, as some reported models based exclusively on demographic (but not physical examination) information have been shown to decrease their performance after external validation12.
The scores integrating clinical with laboratory (ABC-GOALScl) and imaging (ABC-GOALSclx) findings seem to perform better for prediction of admission to an ICU. Both models include values of glucose, lactate dehydrogenase, and oxygenation at admission that have been previously reported to be associated with prognosis in COVID-199,11,33,34. Lactate dehydrogenase has been previously integrated into a predictive model (see below). In contrast to other laboratory tests that may increase during the progression of the disease, such as D-dimer and cardiac troponin levels 35,36, LDH has proved to be the most robust laboratory predictor for admission to ICU when evaluated at first-contact. Glucose and oxygenation were also included in these models. Oxygenation, reflected by the S/F ratio, probably reflects the disease extension along with the imaging findings. It is worth noting that the model that included imaging findings (ABC-GOALSclx) had an equivalent performance to the clinical and laboratory model, therefore, it may not be necessary to perform imaging in a patient who is planned to be referred based on the clinical plus laboratory model (ABC-GOALScl).
We evaluated two scoring models that have been derived to predict the risk of COVID-19 progression: the CALL score24 and the COVID-GRAM score25. Both were derived from Chinese populations. As shown, both scores had lower performance than the ABC-GOALS scores in our population. The CALL score24 includes the presence of any comorbidity, age, lymphocyte count, and LDH levels. Most patients in our study presented with lymphocyte count below the cut-off set in the CALL score, therefore reducing the predictive value of the score and of this specific parameter. For the COVID-GRAM score25, it includes X-ray abnormalities, which were present in all of our patients by lung CT-scan. It also includes rare manifestations of COVID-19 that were barely documented in our population, such as hemoptysis and unconsciousness. Both, the CALL and COVID-GRAM scores, were derived from populations with a lower risk of critical care admission than our population, as 19.2% and 8.2% of the patients developed critical illness, respectively, compared to 34.9% and 47.9% in our development and validation cohorts, respectively. Thus, specific items included in the scores plus the derivation from a population with a lower risk for ICU admission may explain the lower performance of these scores in our population.
We showed that repurposing of other scores employed in pneumonia may not perform adequately to predict the need for admission to ICU in COVID-19 pneumonia. For example, the MuLBSTA score 22 has been validated for mortality prediction in viral pneumonia but may have a lower performance in COVID-19 pneumonia. This score includes the number of affected lobes by viral pneumonia, however, all patients in our study showed multi-lobar lung infiltrates reducing the performance of this parameter. Other variables included in this score, such as bacterial coinfection, may not perform well when evaluated at admission, as bacterial coinfection more frequently occurs later in the disease evolution.
This study has some strengths. First, all data were collected prospectively with very few missing data for all collected variables. Second, there was an appropriate number of patients in the group that required admission to an ICU, which allowed the study of multiple predictors. Third, the study is reported based on TRIPOD guidelines 37
There are limitations to this study. The study included a single-center and very sick population with high rates of ICU admission; therefore, the score may perform well in severe hospitalized patients with COVID-pneumonia. There is still a need for external validation. Another limitation is that medical practice and admission criteria to ICU may vary between institutions and countries, especially in stressed health systems. This score was derived from a population living in Mexico City at >7300 feet over the sea level. The mean partial pressure of oxygen at this altitude has been estimated at around 66mmHg (estimated normal baseline P/F ratio ≈ 314) 38,39. Therefore, we used the S/F ratio to account for respiratory compensation and the right-shift of the hemoglobin dissociation curve that takes place at higher altitudes. Finally, the thresholds set to define the low-, moderate- and high-risk groups should be adapted to local needs and ICU availability. Therefore, we provided the estimated percentages of ICU admission for each sum of the score.
In summary, the ABC-GOALS score represents a tool to evaluate patients with COVID-19 at admission to the ED, designed to timely predict their risk of admission to an ICU. This score may help early referral and planning of attention during the COVID-19 pandemic.
Data Availability
The data included into this manuscript has not yet been subject of peer review.
FUNDING
None.
DECLARATION OF INTEREST
None.
ACKNOWLEDGEMENTS
The authors want to acknowledge all the health care team at Instituto Nacional de Ciencias Médicas y Nutrición Salvador Zubirán at Mexico City for their extraordinary work during this pandemic. We thank Pablo Galindo-Vallejo and Mauricio Moreno-Yañez for the development of the mobile phone application.