Derivation and validation of a prediction rule for sedative-associated delirium during acute respiratory failure requiring mechanical ventilation

Introduction

Delirium, a syndrome of acutely altered attention, awareness, and cognition, affects up to 80% of patients with acute respiratory failure requiring mechanical ventilation.^1-4 Delirium is associated with both ICU and hospital mortality.^5,6 Among survivors, delirium is also associated with long-term cognitive impairment,^2,7 which reduces quality of life and, for many patients,⁸ is a less desirable outcome than death.

During acute respiratory failure, delirium is likely a manifestation of different or converging pathophysiologic processes. Recognizing this multifaceted pathophysiology, clinicians and researchers often subcategorize delirium by probable etiology, with common examples including alcohol withdrawal delirium, septic delirium, delirium associated with hepatic dysfunction (also known as “hepatic encephalopathy”), and medication-induced delirium. In a recent study with a large cohort of ICU patients, 90% of whom were mechanically ventilated, we found that sedative-associated delirium was the most common risk factor-defined delirium subtype.⁹ Like hypoxic and septic delirium, sedative-associated delirium was associated with decreased long-term cognitive function after recovery from critical illness.⁹

In contrast to other etiologies of delirium, sedative-associated delirium is at least in part under the control of the clinical team. However, deliriogenic sedation practices, such as frequent use of benzodiazepines, persist worldwide—indeed, this has worsened since the beginning of the COVID-19 pandemic.¹⁰ Sedative choice and depth is ideally personalized according to each patient’s condition and needs, but evidence-based tools to guide sedation practices are limited to sedation scales used to gauge depth of sedation.^11,12 Therefore, we sought to derive and validate a simple, transparent prediction tool that quantifies the risk of sedative-associated delirium during invasive mechanical ventilation, with a particular focus on the newly intubated patient. If accurate and reliable, this tool could then inform clinicians as they consider risk-benefit tradeoffs regarding sedation, particularly in the immediate post-intubation period.

Methods

Statistical Analysis

We analyzed data using Stata (version 18 StataCorp, College Station, TX) and R (version 4.1.2, R Foundation for Statistical Computing, Vienna, Austria). Missing BRAIN-ICU data were imputed as previously described.² We imputed missing delirium data in the ALIR dataset (for approximately 9% of measurements) using the MICE package for multivariate imputation in R.

We chose an initial 28 candidate variables that had previously been shown to associate with delirium risk.²⁴ Because we sought to derive a relatively simple, transparent score to facilitate implementation, we applied logistic regression so that the resulting model would be intuitively interpretable, the strengths of individual predictors would be explicit, and the model could be used for new prediction in different data sets. Specifically, we used backwards logistic regression with variable elimination at a threshold of p > 0.2 to streamline the model. We used the entire derivation set to create the initial model and used hundred-fold bootstrapping to internally validate the model. Some data (notably frailty and history of stroke) were not captured in the validation cohort, therefore probabilities were calculated in the external validation cohort without those terms. Finally, for both internal and external validation, we assessed discrimination using the area under the receiver-operator curve (i.e., the C-statistic) and assessed calibration graphically and with the Brier index.

Results

Of the 1040 patients in the BRAIN-ICU/MIND-ICU data set, 836 were intubated and mechanically ventilated on the day of enrollment and were used to generate the prediction model. Baseline data are displayed in Table 1. Of these 836 patients, 571 (68%) had sedative-associated delirium in the first 72 hours after study enrollment.

The 28 initial candidate variables we analyzed in the initial regression are shown in Table 2. The strongest predictors were benzodiazepine and propofol doses as well as severity of illness (as measured by SOFA score); other predictors included opioid dose, malignancy, COPD, sepsis at admission, sex, and age (Table 3).

Concordance statistic analysis showed good discrimination, with an area under the receiver-operator curve of 0.83 (Figure 1). Internal validation via hundredfold bootstrapping showed a C-statistic of 0.81 with good graphical calibration (Figure 2). External validation in the ALIR cohorts also showed good discrimination, with a C-statistic of 0.70 (Figure 3).

• Download figure
• Open in new tab

Figure 1.

Receiver-operator characteristic curve for training set.

• Download figure
• Open in new tab

Figure 2.

Calibration plot after internal validation using hundredfold bootstrapping.

• Download figure
• Open in new tab

Figure 3.

Receiver-operator characteristic curve in external validation data set.

Discussion

In this study, we found that a 10-variable clinical prediction tool accurately predicted sedative-associated delirium in two multihospital cohorts of mechanically ventilated acute respiratory failure patients. Variables that are readily available to clinicians—age, admission diagnosis, severity of illness (as measured by the SOFA score), malignancy, COPD, sepsis, sex, and doses of benzodiazepines, propofol, and opioids—successfully predicted sedative-associated delirium during the early period of mechanical ventilation. Future studies are now needed to determine whether a personalized sedation protocol based on this model and other relevant data improves outcomes when used clinically.

The most common, and likely most directly modifiable, cause of delirium during acute respiratory failure is pharmacologic sedation. Despite longstanding guidance by multiple professional societies, much of the sedation provided in ICUs around the world is not concordant with guidelines and often places patients at higher risk of delirium.²⁵ The reasons for lack of concordance are multiple and complex but may often arise from the need to balance the risks of sedative-associated delirium against other risks inherent in the care of mechanically ventilated patients, for example, the occasional need to maintain deep sedation during management of severe hypoxemic respiratory failure.

Even guideline-concordant sedation is not without delirium risk. Sedation is commonly provided during mechanical ventilation for valid clinical reasons—for example, to treat anxiety or agitation or to avoid excessive work of breathing. Until now, clinicians have not had a way to quantify the delirium risk of any given sedation strategy. Our work allows for the estimation of risk and thus the balancing of that risk against competing interests including work of breathing, metabolic demand, and patient agitation. This tool allows a clinician to estimate the risks for a specific patient associated with multiple different sedation options and thereby to incorporate explicit risk information along with the goals of sedation into his or her decision-making. Thus, this sedation-associated delirium prediction tool could be used to guide decision-making about sedation and may serve as a key element in a personalized sedation protocol, the effects of which should be examined in future clinical trials.

Numerous previous studies have examined risk factors for delirium during critical illness,^26-28 but none, to our knowledge, have focused specifically on sedative-associated delirium in mechanically ventilated patients. Additionally, though several delirium risk prediction tools have been generated and validated in ICU cohorts,^29-31 these tools do not distinguish sedative-associated delirium—which is directly related to treatment decisions made by clinicians—from numerous other delirium subtypes and therefore may be less useful as part of a personalized sedation protocol.

Strengths of our study include the use of diverse, high-quality study cohorts for both generation and internal and external validation of the prediction tool. The BRAIN-ICU cohort has been extensively studied and includes data on variables related to patient demographics, comorbidities, acute illness, treatments, and delirium outcomes. The ALIR cohort is a similarly high-quality but fundamentally different cohort, as it is drawn from medical and cardiac intensive care units and includes very few post-surgical patients. The discriminatory power of the model during external validiation thus speaks to its likely generalizability. Other strengths include the tool’s strong face validity and transparency. The predictors included in the final model have all been consistently associated with delirium, and benzodiazepines have been shown to be the highest-risk class of sedative medications used in the intensive care unit, a finding consistent with prior research. One benefit of logistic regression is that it produces clear, transparent effects of each model component. For example, the odds of sedative-associated delirium increase by a factor of 1.55 if ten milligrams of midazolam are used, for example. This transparency should, in theory, make it easier for clinicians to assess the risks of multiple strategies for the same patient and thus to select the lowest-risk strategy that meets the desired sedation goals. Finally, this work is the first to our knowledge to allow for the explicit estimation of sedative-associated delirium risk among mechanically ventilated patients.

One major limitation of this work is the confounding of delirium by coma. Since coma prevents assessment of delirium in the case of a comatose patient, heavy sedation to the point of coma will cause a patient not to score as positive for delirium even if he or she might have been delirious with less sedation. Another limitation of this work include the use of a potentially less predictive statistical approach. More advanced machine learning techniques, such as random forest, extreme gradient boosting, or deep learning neural networks, may fit a given data set better and may generate more accurate predictions. These methods, however, are more prone to overfitting. Additionally, it can be difficult or impossible to interpret the individual variable effects within such models. Along similar lines, the variables available in the two cohorts were not identical; more accurate predictions might be possible with more complete information. A final limitation was inability to include certain other delirium risk factors that were not measured in the cohorts analyzed, such as sensory disturbance or the presence of some invasive devices, such as central venous catheters or fecal collection devices. This limitation could be addressed in prospective validation studies of this model.

Conclusion

We generated a simple, straightforward prediction tool for early sedative-associated delirium in a diverse cohort of mechanically ventilated patients and validated the tool internally and externally. Our findings suggest that sedative-associated delirium in mechanically ventilated ICU patients can be predicted with good confidence. Since our prediction tool allows treating clinicians to predict the risk of delirium in individual patients for any given sedative strategy, it may allow clinicians to personalize sedation based on explicit risk-benefit tradeoffs. This prediction tool should now be optimized and studied prospectively before its implementation as a clinical decision support tool is tested.

Declarations