Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System

Katharine E. Henry, Roy Adams, Cassandra Parent, Anirudh Sridharan, Lauren Johnson, David N. Hager, Sara E. Cosgrove, Andrew Markowski, Eili Y. Klein, Edward S. Chen, Maureen Henley, Sheila Miranda, Katrina Houston, Robert C. Linton II, Anushree R. Ahluwalia, Albert W. Wu, Suchi Saria
doi: https://doi.org/10.1101/2021.07.02.21259941
Katharine E. Henry
1Department of Computer Science, Johns Hopkins University, Baltimore, MD
2Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, MD
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Roy Adams
2Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, MD
3Department of Psychiatry and Behavioral Science, Johns Hopkins School of Medicine, Baltimore, MD
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Cassandra Parent
4Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Anirudh Sridharan
5Howard County General Hospital, Columbia, MD
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lauren Johnson
6Department of Quality Improvement, Johns Hopkins Hospital, Baltimore, MD
MS, RT, CQHP
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
David N. Hager
7Department of Medicine, Johns Hopkins School of Medicine, Baltimore, MD
MD PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sara E. Cosgrove
7Department of Medicine, Johns Hopkins School of Medicine, Baltimore, MD
MD, MS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andrew Markowski
8Suburban Hospital, Bethesda, MD
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Eili Y. Klein
9Department of Emergency Medicine, Johns Hopkins School of Medicine, Baltimore, MD
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Edward S. Chen
7Department of Medicine, Johns Hopkins School of Medicine, Baltimore, MD
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Maureen Henley
6Department of Quality Improvement, Johns Hopkins Hospital, Baltimore, MD
MSN, RN
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sheila Miranda
7Department of Medicine, Johns Hopkins School of Medicine, Baltimore, MD
MS, RN, CCRN
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Katrina Houston
6Department of Quality Improvement, Johns Hopkins Hospital, Baltimore, MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Robert C. Linton II
5Howard County General Hospital, Columbia, MD
MBA MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Anushree R. Ahluwalia
7Department of Medicine, Johns Hopkins School of Medicine, Baltimore, MD
9Department of Emergency Medicine, Johns Hopkins School of Medicine, Baltimore, MD
10Department of Pharmacy, Johns Hopkins School of Medicine, Baltimore, MD
MSN RN
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Albert W. Wu
11Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD
12Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD
13Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: ssaria@cs.jhu.edu
Suchi Saria
1Department of Computer Science, Johns Hopkins University, Baltimore, MD
2Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, MD
7Department of Medicine, Johns Hopkins School of Medicine, Baltimore, MD
11Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD
14Bayesian Health, New York, NY
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: ssaria@cs.jhu.edu
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

ABSTRACT

Machine learning-based clinical decision support tools for sepsis create opportunities to identify at-risk patients and initiate treatments earlier, an important step in improving sepsis outcomes. Increasing use of such systems means quantifying and understanding provider adoption is critical. Using real-time provider interactions with a sepsis early detection tool (Targeted Real-time Early Warning System) deployed at five hospitals over a two-year period (469,419 screened patient encounters, 9,805 (2.1%) of which were retrospectively identified as having sepsis), we found high adoption rates (89% of alerts were evaluated by a physician or advanced practice provider) and an association between use of the tool and earlier treatment of sepsis patients (1.85 (95% CI: 1.66 - 2.00) hour reduction in median time to first antibiotics order). Further, we found that provider-related factors had the strongest association with alert adoption and that case complexity and atypical presentation were associated with dismissal of alerts on sepsis patients. Beyond improving the performance of the system, efforts to improve adoption should focus on provider knowledge, experience, and perceptions of the system.

Competing Interest Statement

Under a license agreement between Bayesian Health and the Johns Hopkins University, Dr. Henry, Dr. Saria, and Johns Hopkins University are entitled to revenue distributions. Additionally, the University owns equity in Bayesian Health. This arrangement has been reviewed and approved by the Johns Hopkins University in accordance with its conflict of interest policies. Dr. Saria also has grants from Gordon and Betty Moore Foundation, the National Science Foundation, the National Institutes of Health, Defense Advanced Research Projects Agency, the Food and Drug Administration, and the American Heart Association; she is a founder of and holds equity in Bayesian Health; she is the scientific advisory board member for PatientPing; and she has received honoraria for talks from a number of biotechnology, research, and health-tech companies. This arrangement has been reviewed and approved by the Johns Hopkins University in accordance with its conflict of interest policies. Dr. Hager discloses salary support and funding to his institution from the Marcus Foundation for the conduct of the Vitamin C, Thiamine, and Steroids in Sepsis Trial. Dr. Cosgrove consulting fees from Basilea for work on an infection adjudication committee for a S. aureus bacteremia trial. The other authors declare no disclosures of conflicts of interest.

Funding Statement

The authors gratefully acknowledge the following sources of funding: the Gordon and Betty Moore Foundation (award #3926), the National Science Foundation Future of Work at the Human-technology Frontier (award #1840088), and the Alfred P. Sloan Foundation research fellowship (2018).

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

This study was approved by the Johns Hopkins University internal review board (IRB No. 00252594) and a waiver of consent was obtained.

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Data Availability

The data is not available for public use.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-ND 4.0 International license.
Back to top
PreviousNext
Posted July 07, 2021.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System
Katharine E. Henry, Roy Adams, Cassandra Parent, Anirudh Sridharan, Lauren Johnson, David N. Hager, Sara E. Cosgrove, Andrew Markowski, Eili Y. Klein, Edward S. Chen, Maureen Henley, Sheila Miranda, Katrina Houston, Robert C. Linton II, Anushree R. Ahluwalia, Albert W. Wu, Suchi Saria
medRxiv 2021.07.02.21259941; doi: https://doi.org/10.1101/2021.07.02.21259941
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Evaluating Adoption, Impact, and Factors Driving Adoption for TREWS, a Machine Learning-Based Sepsis Alerting System
Katharine E. Henry, Roy Adams, Cassandra Parent, Anirudh Sridharan, Lauren Johnson, David N. Hager, Sara E. Cosgrove, Andrew Markowski, Eili Y. Klein, Edward S. Chen, Maureen Henley, Sheila Miranda, Katrina Houston, Robert C. Linton II, Anushree R. Ahluwalia, Albert W. Wu, Suchi Saria
medRxiv 2021.07.02.21259941; doi: https://doi.org/10.1101/2021.07.02.21259941

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Health Informatics
Subject Areas
All Articles
  • Addiction Medicine (271)
  • Allergy and Immunology (555)
  • Anesthesia (135)
  • Cardiovascular Medicine (1766)
  • Dentistry and Oral Medicine (238)
  • Dermatology (173)
  • Emergency Medicine (313)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (661)
  • Epidemiology (10805)
  • Forensic Medicine (8)
  • Gastroenterology (593)
  • Genetic and Genomic Medicine (2957)
  • Geriatric Medicine (288)
  • Health Economics (534)
  • Health Informatics (1932)
  • Health Policy (836)
  • Health Systems and Quality Improvement (745)
  • Hematology (293)
  • HIV/AIDS (632)
  • Infectious Diseases (except HIV/AIDS) (12521)
  • Intensive Care and Critical Care Medicine (696)
  • Medical Education (300)
  • Medical Ethics (86)
  • Nephrology (324)
  • Neurology (2803)
  • Nursing (151)
  • Nutrition (433)
  • Obstetrics and Gynecology (559)
  • Occupational and Environmental Health (597)
  • Oncology (1472)
  • Ophthalmology (444)
  • Orthopedics (172)
  • Otolaryngology (257)
  • Pain Medicine (190)
  • Palliative Medicine (56)
  • Pathology (381)
  • Pediatrics (868)
  • Pharmacology and Therapeutics (366)
  • Primary Care Research (338)
  • Psychiatry and Clinical Psychology (2645)
  • Public and Global Health (5379)
  • Radiology and Imaging (1015)
  • Rehabilitation Medicine and Physical Therapy (597)
  • Respiratory Medicine (726)
  • Rheumatology (330)
  • Sexual and Reproductive Health (289)
  • Sports Medicine (279)
  • Surgery (327)
  • Toxicology (47)
  • Transplantation (150)
  • Urology (126)