Abstract
Importance Late predictions of hospitalized patient deterioration, resulting from early warning systems (EWS) with limited data sources and/or a care team’s lack of shared situational awareness, contribute to delays in clinical interventions. The COmmunicating Narrative Concerns Entered by RNs (CONCERN) Early Warning System (EWS) uses real-time nursing surveillance documentation patterns in its machine learning algorithm to identify patients’ deterioration risk up to 42 hours earlier than other EWSs.
Objective To test our a priori hypothesis that patients with care teams informed by the CONCERN EWS intervention have a lower mortality rate and shorter length of stay (LOS) than the patients with teams not informed by CONCERN EWS.
Design One-year multisite, pragmatic controlled clinical trial with cluster-randomization of acute and intensive care units to intervention or usual-care groups.
Setting Two large U.S. health systems.
Participants Adult patients admitted to acute and intensive care units, excluding those on hospice/palliative/comfort care, or with Do Not Resuscitate/Do Not Intubate orders.
Intervention The CONCERN EWS intervention calculates patient deterioration risk based on nurses’ concern levels measured by surveillance documentation patterns, and it displays the categorical risk score (low, increased, high) in the electronic health record (EHR) for care team members.
Main Outcomes and Measures Primary outcomes: in-hospital mortality, LOS; survival analysis was used. Secondary outcomes: cardiopulmonary arrest, sepsis, unanticipated ICU transfers, 30-day hospital readmission.
Results A total of 60 893 hospital encounters (33 024 intervention and 27 869 usual-care) were included. Both groups had similar patient age, race, ethnicity, and illness severity distributions. Patients in the intervention group had a 35.6% decreased risk of death (adjusted hazard ratio [HR], 0.644; 95% confidence interval [CI], 0.532-0.778; P<.0001), 11.2% decreased LOS (adjusted incidence rate ratio, 0.914; 95% CI, 0.902-0.926; P<.0001), 7.5% decreased risk of sepsis (adjusted HR, 0.925; 95% CI, 0.861-0.993; P=.0317), and 24.9% increased risk of unanticipated ICU transfer (adjusted HR, 1.249; 95% CI, 1.093-1.426; P=.0011) compared with patients in the usual-care group.
Conclusions and Relevance A hospital-wide EWS based on nursing surveillance patterns decreased in-hospital mortality, sepsis, and LOS when integrated into the care team’s EHR workflow.
Trial Registration ClinicalTrials.gov Identifier: NCT03911687 https://clinicaltrials.gov/ct2/show/NCT03911687
Question Do patients whose care team receive the CONCERN Early Warning System (EWS) intervention have a lower mortality rate and shorter length of stay than patients in the usual-care group?
Findings In this multisite, pragmatic cluster-randomized controlled clinical trial that included 60 893 hospital patient encounters, patients whose care team received the CONCERN EWS intervention had a 35.6% decreased risk of death and 11.2% shorter length of stay compared with those in the usual-care group.
Meaning A machine learning-based EWS modeled on nursing surveillance patterns significantly decreased the risk of inpatient deterioration events.
Competing Interest Statement
Dr. Bates reports grants and personal fees from EarlySense, personal fees from CDI Negev, equity from ValeraHealth, equity from Clew, equity from MDClone, personal fees and equity from AESOP, personal fees and equity from FeelBetter, personal fees and equity from Guided Clinical Solutions and grants from IBM Watson Health, outside the submitted work.
Clinical Trial
NCT03911687
Clinical Protocols
https://pubmed.ncbi.nlm.nih.gov/34889766/
Funding Statement
This study was funded by the National Institute of Nursing Research (NINR 1R01NR016941, COmmunicating Narrative Concerns Entered by RNs (CONCERN): Clinical Decision Support Communication for Risky Patient States) and Reducing Health Disparities Through Informatics (T32NR007969). The funder of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Institutional review boards of Columbia University and Brigham and Women's Hospital gave ethical approval for this work.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
Individual participant data including data dictionaries and study protocol will be made available. Specifically, de-identified individual shift level and encounter level data for primary and secondary outcomes in this trial will be made available after de-identification. Data will be made available within 6 months following article publication with no anticipated end date. We will make the data available to investigators whose proposed use of the data has been approved by an independent review board. Data will be available through our study website https://www.dbmi.columbia.edu/concern-study/. To gain access, data requestors will need to sign a data access agreement. We will share statistical analysis code upon request and additional information, including study protocol details, is available in our online supplement. The CONCERN EWS algorithm is considered intellectual property that will not be shared but is described in our online supplement.