In an influenza pandemic, natural disaster, or human-made disaster, a mass influx of critically ill patients could overwhelm existing intensive care unit (ICU) resources. In this setting it would be helpful to predict which patients in the ICU would not survive despite full care and which patients require ongoing critical care resources, such as mechanical ventilators, so that equipment and personnel needs are anticipated. It may also be necessary to triage patients already in the ICU to reserve resources for those most likely to benefit from them. Useful triage tools would predict which patients have the greatest chance of survival, which patients are likely to die, and which patients will require mechanical ventilation. These are important issues not only for the initial allocation of resources but also for reallocation of resources over time to support the ongoing needs of the survivors of critical illness. The goals of this study are to evaluate the Sequential Organ Failure Assessment (SOFA) score and a simpler Modified SOFA (MSOFA) for their ability to predict mortality and the need for mechanical ventilation in critically ill patients after admission to the ICU.
The SOFA score is a validated measure of organ failure over time and a predictor of mortality in critically ill patientsReference Ferreira, Bota, Bross, Mélot and Vincent1Reference Vincent, Moreno and Takala2Reference Cabré, Mancebo and Solsona3 that has been incorporated into triage protocols for critical care in the event of an influenza pandemic or a mass influx of patients during a disaster.Reference Hick and O’Laughlin4Reference Christian, Hawryluck and Wax5Reference Powell, Christ and Birkhead6Reference Devereaux, Dichter and Christian7 The SOFA score combines a clinical assessment of 2 organ systems, cardiovascular system and central nervous system, with laboratory measurements for evaluation of 4 other organ systems: respiratory, hematologic, liver, and renal (Table 1). The requirement for arterial and venous blood specimens from each patient to calculate a SOFA score may prove impractical with a large number of patients and constrained resources. Given this limitation, we created the simpler MSOFA score8 (Table 2) to minimize reliance on laboratory resources. The MSOFA score eliminates the platelet count, replaces partial pressure of arterial oxygen (PaO 2) with arterial oxygen saturation measured by a pulse oximeter (SpO2), and replaces serum bilirubin with clinical assessment of scleral icterus or jaundice (Table 3). The only laboratory value required for the MSOFA is creatinine, which can be measured using a bedside point-of-care testing device.
TABLE 1 Sequential Organ Failure Assessment ScoreReference Ferreira, Bota, Bross, Mélot and Vincent1Reference Vincent, Moreno and Takala2
TABLE 2 Modified Sequential Organ Failure Assessment Score
TABLE 3 Comparison of the Sequential Organ Failure Assessment and Modified Sequential Organ Failure Assessment Scores
We hypothesized that the MSOFA score would predict mortality and need for mechanical ventilation as well as the SOFA score and thus could be used in triage protocols in resource-constrained critical care environments during a disaster or pandemic. We initially compared the MSOFA to the SOFA in a preliminary study using retrospective data from patients admitted to a 12-bed medical, surgical, and trauma ICU at LDS Hospital in Salt Lake City, Utah, during the 2006 calendar year. On the basis of this analysis, we prospectively compared the MSOFA with the SOFA in patients admitted to a 24-bed medical, surgical, and trauma ICU at Intermountain Medical Center in Murray, Utah, during the 2008 calendar year.
METHODS
In developing the MSOFA score, we identified bedside clinical parameters and electronic medical record data that could function as surrogates for the evaluation of respiratory and liver system organ function in the SOFA score. The cardiovascular, central nervous system, and renal scores for the MSOFA score remained the same as the SOFA score (Tables 1 to 3). We eliminated the hematologic system from the MSOFA score so that 5 organ systems are scored on a 0-to-4 scale according to specified criteria indicating severity, with a maximum score of 19 (Table 2). For the SOFA score, 6 organ systems are scored with a maximum score of 24 (Table 1).
To calculate the respiratory component of the MSOFA score, the SpO2 is divided by the fraction of inspired oxygen (FiO2) to calculate an SpO2/FiO2 ratio. For patients receiving nasal flow oxygen, we estimated FiO2 by multiplying the liter flow per minute by 0.03 and adding that to 0.21. The SpO2/FiO2 ratio has been validated separately as a surrogate for the PaO 2/FiO2 ratio that is used by the SOFA score.Reference Rice, Wheeler, Bernard, Hayden, Schoenfeld and Ware9Reference Pandharipande, Shintani and Hagerman10 The liver portion of the MSOFA score requires the observation of scleral icterus or jaundice documented by the bedside nurse.
MSOFA and SOFA scores were determined from parameters at the end of 24-hour time periods relative to ICU admission. For day 1 the time period was 6 hours before ICU admission to 18 hours after, for day 3 the time period was 48 to 72 hours after ICU admission, and for day 5 the time period was 96 to 120 hours after ICU admission. Scores were calculated only for patients who were alive and in the ICU for the entire duration of the day 1, day 3, or day 5 scoring period. The SOFA and MSOFA score on day 1 included only those patients who had survived the entire initial 18 hours in the ICU. The SOFA and MSOFA score on days 3 and 5 included only those patients who had survived at least 72 hours and 120 hours, respectively, and remained in the ICU at the end of the scoring period. We used only laboratory measurements that were obtained for the care of the patient during the specified time periods for the calculation of the MSOFA or SOFA scores. We selected the worst values for each parameter in the specified time period, except for the Glasgow Coma Scale (GCS), as was done in the primary studies on the SOFA score.Reference Ferreira, Bota, Bross, Mélot and Vincent1 For the GCS, we selected the maximum score in the 24-hour period. In the primary study on the SOFA, the presumed GCS score was used for sedated patients.Reference Ferreira, Bota, Bross, Mélot and Vincent1 Our standard protocol is to chart the actual GCS score determined regardless of sedation or paralysis; therefore, to evaluate the contribution of the GCS to the SOFA score, we also analyzed our data using the minimum GCS score in 24 hours for comparison. For a missing value, we used the closest available value up to 4 hours before the 24-hour scoring interval. Where no value was available, the predictor was assumed to be normal and assigned a score of 0.
LDS Hospital and the Intermountain Medical Center are both Intermountain Healthcare Hospitals and have mature electronic clinical information systems that include electronic charting of scleral icterus and jaundice, which allowed us to retrospectively determine the MSOFA and SOFA scores on ICU admission for every patient admitted to the LDS Hospital Shock Trauma Respiratory ICU in 2006. For the prospective study of the MSOFA and SOFA scores in the Shock Trauma ICU at Intermountain Medical Center, we instructed the clinical nursing staff of the study and specifically trained them to document jaundice consistently within the electronic medical record. All of the other components of the MSOFA and SOFA scores that required clinical evaluation were part of standard nursing bedside charting. The institutional review board of LDS Hospital and the Intermountain Medical Center, Intermountain Healthcare, approved this observational study with waiver of informed consent.
The endpoints used for validation of the SOFA and MSOFA scores were 30-day all-cause mortality and need for mechanical ventilation on days 3 and 5. Mechanical ventilation was prospectively recorded for all of the patients and the need for ventilation was defined as any receipt of mechanical ventilation on the relevant study day.
Statistical Methods
We compared MSOFA score mortality prediction to the SOFA score by areas under the receiver operating characteristic curve (AUC) using the technique of DeLong and colleaguesReference DeLong, DeLong and Clarke-Pearson11 and calculated an exact binomial confidence interval for the AUC. In the 2006 retrospective study, the MSOFA score performed well, but the evaluation revealed the need to standardize charting of jaundice by the nursing staff when they identified a patient with jaundice. This was done in 2007 and the prospective evaluation began in 2008.
To investigate the relevance of the evolution of organ dysfunction to mortality prediction for patients with scores on days 1, 3, and 5 during their ICU stay, we evaluated several summarization and trending transformations in a logistic regression model for both MSOFA and SOFA. For the day 3 model, we evaluated quantitative and qualitative change, summed scores, maximum score, and a linear prediction of the day 5 score. For the day 5 model, we included everything from the day 3 model (incorporating the day 1 and day 3 scores only, the day 3 and day 5 scores only, and all 3 days' scores). In addition, we fitted a nonlinear prediction model for day 5 to the available scores. The utility of adding additional terms to the model was evaluated within a forward stepwise bootstrap procedure. The “out of the bag” sample was used to avoid a biased estimate of AUC due to overfitting.Reference Efron and Tibshirani12Reference Harrell13 The most recent MSOFA was always selected first. Additional predictors were maintained in the model only if they significantly increased the AUC. A subsequent 1000-bootstrap sample estimate was used to confirm these findings.
In a secondary analysis we evaluated the day 1 scores as predictors of the receipt of mechanical ventilation on days 3 and 5, comparing AUC using the technique of DeLong et alReference DeLong, DeLong and Clarke-Pearson11 and calculating exact binomial confidence intervals for the AUC. Analyses were performed using the R statistical package, version 2.9.1 (R Foundation for Statistical Computing, Vienna, Austria) and MedCalc statistical software, version 11.3.8 (MedCalc Software, Mariakerke, Belgium).
RESULTS
The retrospective derivation cohort in 2006 included 718 patients (58% male patients) with a 30-day inpatient mortality of 17.3%. In the 2006 cohort, day 1 SOFA and MSOFA scores predicted 30-day mortality with an AUC of 0.78 (95% confidence interval [CI] 0.74-0.82) and 0.77 (95% CI 0.73-0.82), respectively (P > .2 for comparison).
In 2008, 1803 patients (56% male patients) were admitted to the Shock Trauma ICU with a mortality of 12.1%. Of those 1803 patients, 33 patients died during the day 1 scoring period and were excluded from the analysis. The prospective study cohort in 2008 included 1770 patients (56% male patients) with a 30-day mortality of 10.5%. Table 4 compares the 2006 preliminary study retrospective cohort and the 2008 prospective cohort. Figure 1 shows the flow of patients admitted to the Shock Trauma ICU in 2008 and the number of patients included in the 2008 prospective days 1, 3, and 5 analyses. Table 5 shows the percentage of missing values for parameters collected for organ system scoring in calculating the SOFA and MSOFA scores during each scoring period. Table 6 shows the characteristics of survivors vs nonsurvivors in the 2008 prospective study including differences in SOFA and MSOFA scores.
FIGURE 1 Flowchart: Number of patients included in each scoring period.
TABLE 4 Comparison of 2006 and 2008 Groups
TABLE 5 Missing Values for Parameters Used in Organ System Scores for SOFA and MSOFA
TABLE 6 Comparison of Survivors and Nonsurvivors in the 2008 Prospective Study
In the 1770 patients alive at the end of the day 1 scoring period, the SOFA score calculated using the maximum GCS gave a greater AUC (0.83, 95% CI 0.81-0.85) as compared to the SOFA score calculated using the minimum GCS score (AUC 0.81, 95% CI 0.79-0.83; P < .001). The day 1 MSOFA score calculated using the maximum GCS score gave a greater AUC (0.84, 95% CI 0.82-0.85) as compared to the MSOFA score calculated using the minimum GCS score (AUC 0.82, 95% CI 0.80-0.84; P < .001). Because the SOFA and MSOFA scores calculated using the maximum GCS score on day 1 performed better than the SOFA and MSOFA scores calculated using the minimum GCS score, we used the SOFA and MSOFA scores calculated using the maximum GCS score in all further analyses.
There was no difference between the day 1 SOFA and MSOFA scores (P = .33) for predicting mortality (Figure 2). For day 3, SOFA and MSOFA scores performed similarly well at predicting mortality for the 828 patients remaining in the ICU, with an AUC of 0.78 (95% CI 0.75-0.81) and 0.79 (95% CI 0.76-0.82), respectively (P = .45). For day 5, SOFA and MSOFA scores performed less well at predicting mortality on the 369 patients remaining in the ICU, with an AUC of 0.72 (95% CI 0.67-0.76) and 0.74 (95% CI 0.70-0.79), respectively (P = .21). None of the novel predictors added significantly to the AUC for the day 1, 3, or 5 predictions.
FIGURE 2 A, Mortality by day 1 Sequential Organ Failure Assessment and Modified Sequential Organ Failure Assessment scores for prospectively collected data of patients admitted to the Shock Trauma ICU at Intermountain Medical Center during the 2008 calendar year. B, Analysis of area under the receiver operating curve for day 1 Sequential Organ Failure Assessment and Modified Sequential Organ Failure Assessment prediction of mortality in the 2008 cohort.
The day 1 SOFA and MSOFA scores predicted the need for mechanical ventilation on day 3 with an AUC of 0.83 (95% CI 0.81-0.84) and 0.82 (95% CI 0.80-0.84), respectively (P = .43). Of the 828 patients remaining in the ICU on day 3, 202 (24%) were receiving mechanical ventilation. The day 1 SOFA and MSOFA scores predicted the need for mechanical ventilation on day 5 with an AUC of 0.76 (95% CI 0.74-0.78) for both (P = .81). Of the 369 patients remaining in the ICU on day 5, 136 (37%) were receiving mechanical ventilation.
Previously published pandemic influenza triage protocols use threshold values of SOFA scores >11 for exclusion from ICU admission, 8 to 11 for intermediate priority, and <8 for high priority for ICU admission.Reference Hick and O’Laughlin4Reference Christian, Hawryluck and Wax5Reference Powell, Christ and Birkhead6 Mortality for threshold levels of MSOFA and SOFA scores are shown in Figure 2A. Day 1 SOFA scores in the 2008 study had 1450 patients with a score of 0 to 7 and a mortality of 5% (70 patients), 216 patients with a score of 8 to 11 and a mortality of 28% (60 patients), and 104 patients with a score of >11 and a mortality of 53% (55 patients). Day 1 MSOFA scores in the 2008 study had 1440 patients with a score of 0 to 7 and a mortality of 4% (64 patients), 259 patients with a score of 8 to 11 and a mortality of 31% (80 patients), and 71 patients with a score of >11 and a mortality of 58% (41 patients).
We performed a subgroup analysis for prediction of mortality from day 1 SOFA or MSOFA score for medical patients (Table 4, n = 991, 13% mortality), postoperative surgical patients (n = 244, 4% mortality), and trauma patients (n = 495, 8% mortality). Among the medical patients, day 1 SOFA and MSOFA scores predicted mortality with an AUC of 0.82 (95% CI 0.79-0.84) and 0.82 (95% CI 0.80-0.85), respectively (P = .59). In the postoperative surgical patients, day 1 SOFA and MSOFA scores predicted mortality with an AUC of 0.70 (95% CI 0.64-0.76) and 0.84 (95% CI 0.79-0.88), respectively (P = .024). In the trauma patients, day 1 SOFA and MSOFA scores predicted mortality with an AUC of 0.87 (95% CI 0.84-0.90) and 0.84 (95% CI 0.80-0.87), respectively (P = .03).
DISCUSSION
Future H1N1 influenza pandemics,Reference Chowell, Bertozzi and Colchero14Reference Perez-Padilla, de la Rosa-Zamboni and Ponce de Leon15 outbreaks of avian influenza,Reference Abdel-Ghafar, Chotpitayasunondh and Gao16Reference Arabi, Gomersall, Ahmed, Boynton and Memish17Reference Beigel, Farrar and Han18Reference Gruber, Gomersall and Joynt19 natural disasters,Reference Devereaux, Dichter and Christian7 or human-made disasters may overwhelm critical care capacities. The number of available mechanical ventilators and ICU beds may not be sufficient for the number of people who are affected.Reference Bartlett20Reference Talmor, Jones, Rubinson, Howell and Shapiro21Reference Carr, Addyson and Kahn22 Understanding which patients are most likely to benefit from receiving critical care resources may allow for triage of patients in a manner that provides the highest likelihood of survival for the most people. Complex mortality prediction models are labor and resource intensive and are difficult to apply in a mass patient influx.Reference Le Gall, Lemeshow and Leleu23Reference Lemeshow and Le Gall24 Algorithms specific to mass casualty trauma triageReference Garner, Lee, Harrison and Schultz25 are not generalizable for a mass influx of influenza patients or for the general population of nontrauma critically ill patients who will compete for critical care resources during a pandemic. A simple triage scoring system focusing on age, respiratory failure, and shock index for use during epidemics has been validated in a cohort of emergency department patients admitted to the hospital with nonsurgical infection and performed moderately well at predicting mortality (AUC 0.73) or need for mechanical ventilation (AUC 0.68),Reference Talmor, Jones, Rubinson, Howell and Shapiro21 but restriction of the cohort to infected patients limits generalizability to triage of acutely injured patients.
The SOFA score has been recommended as a tool in critical care triageReference Hick and O’Laughlin4Reference Christian, Hawryluck and Wax5Reference Powell, Christ and Birkhead6Reference Devereaux, Dichter and Christian7Reference Christian, Joynt, Hick, Colvin, Danis and Sprung26 because of ease of calculation, if the required laboratory tests are available, and validation takes place in a wide variety of critically ill patients.Reference Ferreira, Bota, Bross, Mélot and Vincent1Reference Vincent, Moreno and Takala2Reference Cabré, Mancebo and Solsona3 Reliance on multiple laboratory values, however, may make application impractical during mass critical care triage. We showed that an equivalent mortality prediction can be performed with an MSOFA score using variables easily collected at the bedside. A much smaller study of 144 surgical ICU patients in Indonesia suggested the utility of the MSOFA with an equivalent mortality prediction to the SOFA score.Reference Halim, Murni and Redjeki27 In our prospective study, the MSOFA score predicted mortality with a high AUC of 0.84 that was equivalent to that of the SOFA score. The MSOFA score also performed well in predicting mortality after 3 days of ICU stay, with an AUC of 0.79, again equivalent to the SOFA score. Prediction of mortality at day 5 of ICU stay in 369 critically ill patients was less robust for both the MSOFA and SOFA scores, which is a limitation for application of the SOFA and MSOFA to predict outcomes during the ICU stay. Both the SOFA and MSOFA scores have been proposed as tools to retriage patients during their ICU course, and the degradation of predictive utility over time may have implications for published protocols.Reference Hick and O’Laughlin4Reference Christian, Hawryluck and Wax5Reference Devereaux, Dichter and Christian78 Beyond mortality, the initial SOFA and MSOFA scores both predicted ongoing requirements for mechanical ventilation with no clear difference between them.
In the subgroup analysis separating medical, surgical, and trauma patients, the SOFA and MSOFA performed equally well at predicting mortality in medical patients (AUC 0.82 for both). The MSOFA (AUC 0.84) performed much better at predicting mortality than the SOFA (AUC 0.70) in the postoperative surgical patients (P = .024), but the mortality was low (4%) and the number of patients (244) was small, so the significance is unclear. In the larger group of trauma patients (n = 495) with an 8% mortality, both the SOFA and MSOFA performed well with AUC of 0.87 and 0.84, respectively (P = .03). Medical and trauma patients are those most relevant to mass casualty and triage.
A convenient result of our study is that the MSOFA and SOFA scores directly correlate in the range of the thresholds used in pandemic influenza triage plans in which a score of >11 excludes a patient from critical care, whereas a score of 8 to 11 is an intermediate priority and <8 is a high priority for critical care.Reference Hick and O’Laughlin4Reference Christian, Hawryluck and Wax5 The MSOFA score predicts mortality in these categories equally as well as the SOFA score at the relevant thresholds of 8 and 11 (Figure 2). The SOFA and MSOFA, however, have different scoring ranges (0-24 for SOFA and 0-19 for MSOFA) and it will require further study to confirm that they can be used interchangeably. The threshold MSOFA and SOFA score of >11 used to exclude patients from critical care includes only about 6% of total patients in our cohort, and up to 40% of the excluded patients would survive in a setting where ample critical care resources were available. This emphasizes that these scoring systems can only be 1 part of an overall system of triage. We specifically did not evaluate the effect of significant comorbidities on triage, which is included in published triage protocols that use SOFA or MSOFA.Reference Christian, Hawryluck and Wax58Reference Christian, Joynt, Hick, Colvin, Danis and Sprung26
A recent study emphasized that the practical implementation of a triage system that includes a SOFA score requires training for those who will act as triage officers and protocol modification to minimize the exclusion from critical care of patients who may in fact benefit.Reference Christian, Hamielec and Lazar28 Findings in that same study suggested that application of a SOFA score–based critical care triage protocol could direct scarce resources to patients who are most likely to benefit from them while increasing resource availability.Reference Christian, Hamielec and Lazar28 Two other recent studies, however, have raised concern about using a SOFA score–based critical care triage. One small study evaluating application of the SOFA score retrospectively for critical care triage to 8 patients with 2009 H1N1 influenza found that 5 survivors would have been candidates for withdrawal of life support at 48 hours according to a SOFA score triage.Reference Khan, Hulme and Sherwood29 A larger study applying SOFA score triage criteria to a mix of 255 patients admitted to an ICU found that 116 patients with 39% survival would have been denied admission.Reference Guest, Tantam, Donlin, Tantam, McMillan and Tillyard30
The results of these studies and our study suggest that further evaluation and public discussion is necessary to refine a plan for the allocation of scarce resources and life-sustaining treatments during a public health emergency with a mass influx of critically ill or injured patients.Reference Christian, Sandrock, Devereaux, Geiling, Amundson and Rubinson31Reference White, Katz, Luce and Lo32Reference Fink33
The results from our study and othersReference Christian, Hamielec and Lazar28Reference Khan, Hulme and Sherwood29Reference Guest, Tantam, Donlin, Tantam, McMillan and Tillyard30 suggest that a large number of critically ill patients who have SOFA or MSOFA scores >11 may survive in a mass critical care patient influx, such as an influenza pandemic, and require ongoing care and resource reallocation. For example, patients critically ill with pandemic influenza and SOFA scores >11 may be delegated in a triage system to receive a freestanding continuous positive airway pressure system, rather than positive pressure mechanical ventilation, with an FiO2 of 1.0 to conserve mechanical ventilators for those who have the greatest chance of survival. Some of those patients delegated to continuous positive airway pressure will survive and may have SOFA or MSOFA scores that improve and assign them a higher priority than other critically ill patients for allocation of resources. Scenarios such as this include not only resource allocation but also resource reallocation, and will require further study and careful planning.
A limitation of our study is the focus on triage of patients after, rather than before, admission to the ICU. Determination of which critically ill patients should be admitted to an ICU is an important component of mass critical care triage. The SOFA score has been applied to initial critical care triage in some published triage protocols,Reference Hick and O’Laughlin4Reference Christian, Hawryluck and Wax5Reference Powell, Christ and Birkhead6Reference Devereaux, Dichter and Christian7 even though the SOFA score has only been validated as a severity of illness score for patients in the ICU.Reference Ferreira, Bota, Bross, Mélot and Vincent1Reference Vincent, Moreno and Takala2 The MSOFA score performed similarly to the SOFA score in our study, suggesting that the MSOFA may also be useful for initial triage for admission to the ICU from the emergency department. The day 1 data in our study reflect the acute presentation of critically ill patients because the majority of admissions to our ICU come through the emergency department (75%), and the time spent in the emergency department for patients admitted to the Shock Trauma ICU during 2008 was brief: 3.7 ± 2.0 hours. Further study is warranted to determine the utility of the SOFA or MSOFA scores for triage of critically ill patients in the emergency department for admission to the ICU.
CONCLUSIONS
The MSOFA score predicts mortality in critically ill patients as effectively as the SOFA score and is easier to implement in resource-constrained or pandemic settings. The mortality in the highest scoring group with an SOFA or MSOFA of >11 was only 53% to 58%, which means that implementation of either the MSOFA or SOFA score as a triage tool during a mass influx of critically ill patients would exclude a significant number of patients who would survive with usual critical care resources. Our findings are consistent with recommendations that a triage tool, including the SOFA or MSOFA score, should be applied only when critical care resources are overwhelmed and all of the efforts to obtain additional resources have been institutedReference Devereaux, Dichter and Christian7Reference Christian, Joynt, Hick, Colvin, Danis and Sprung26 and should be combined with criteria that exclude patients based on comorbid conditions that increase mortality.Reference Christian, Hawryluck and Wax58 These are serious issues for consideration and warrant further study and careful ethical and social discussion.
Author Disclosures: The author reports no conflicts of interest.
Acknowledgements The authors thank the Shock Trauma Intensive Care Unit attending staff at Intermountain Medical Center for their support of this study: Drs Don VanBoerum, Tom White, Steve Granger, Frank Thomas, Ellie Hirshberg, and Nathan Dean. They also thank the members of the Utah Hospital Association Pandemic Influenza Triage Committee chaired by Dr Brent Wallace, chief medical officer of Intermountain Healthcare.
