Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Emergency medicine patient wait time multivariable prediction models: a multicentre derivation and validation study

View ORCID ProfileKatie Walker, Jirayus Jiarpakdee, Anne Loupis, Chakkrit Tantithamthavorn, Keith Joe, Michael Ben-Meir, Hamed Akhlaghi, Jennie Hutton, Wei Wang, Michael Stephenson, Gabriel Blecher, Paul Buntine, Amy Sweeny, Burak Turhan
doi: https://doi.org/10.1101/2021.03.19.21253921
Katie Walker
1Emergency Department, Cabrini, Melbourne, Australia
2Cabrini Institute, Cabrini, Melbourne, Australia
3School of Clinical Sciences, Monash University, Melbourne, Australia
4Casey Hospital Emergency Department, Monash Health, Melbourne, Australia
5School of Clinical Sciences at Monash Health, Monash University, Melbourne, Australia
Roles: emergency physician
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Katie Walker
  • For correspondence: katie_walker01@yahoo.com.au
Jirayus Jiarpakdee
6Department of Software Systems and Cybersecurity, Monash University, Melbourne, Australia
Roles: data scientist
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Anne Loupis
2Cabrini Institute, Cabrini, Melbourne, Australia
3School of Clinical Sciences, Monash University, Melbourne, Australia
Roles: project manager
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Chakkrit Tantithamthavorn
6Department of Software Systems and Cybersecurity, Monash University, Melbourne, Australia
Roles: data scientist
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Keith Joe
1Emergency Department, Cabrini, Melbourne, Australia
7Monash Art, Design and Architecture, Monash University, Melbourne, Australia
Roles: emergency physician
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michael Ben-Meir
1Emergency Department, Cabrini, Melbourne, Australia
8School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia
Roles: emergency physician
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hamed Akhlaghi
9Emergency Department; St Vincent’s Hospital, Melbourne, Australia
Roles: emergency physician
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jennie Hutton
9Emergency Department; St Vincent’s Hospital, Melbourne, Australia
10Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, Australia
Roles: emergency physician
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Wei Wang
2Cabrini Institute, Cabrini, Melbourne, Australia
11Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
Roles: statistician
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michael Stephenson
12Ambulance Victoria, Melbourne, Australia
13Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, Australia
14Department of Community Emergency Health and Paramedic Practice, Monash University, Melbourne, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Gabriel Blecher
1Emergency Department, Cabrini, Melbourne, Australia
5School of Clinical Sciences at Monash Health, Monash University, Melbourne, Australia
15Emergency Department Monash Medical Centre, Monash Health, Melbourne, Australia
Roles: emergency physician
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Paul Buntine
16Box Hill Hospital Emergency Department, Eastern Health, Melbourne, Australia
17Eastern Health Clinical School, Monash University, Melbourne, Australia
Roles: emergency physician
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Amy Sweeny
18Emergency Department, Gold Coast University Hospital, Southport, Australia
19Faculty of Health Sciences and Medicine, Bond University, Robina, Australia
Roles: epidemiologist
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Burak Turhan
6Department of Software Systems and Cybersecurity, Monash University, Melbourne, Australia
20Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland
Roles: data scientist
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

Abstract

Objective Patients, families and community members would like emergency department wait time visibility. This would improve patient journeys through emergency medicine. The study objective was to derive, internally and externally validate machine learning models to predict emergency patient wait times that are applicable to a wide variety of emergency departments.

Methods Twelve emergency departments provided three years of retrospective administrative data from Australia (2017-19). Descriptive and exploratory analyses were undertaken on the datasets. Statistical and machine learning models were developed to predict wait times at each site and were internally and externally validated. Model performance was tested on COVID-19 period data (January to June 2020).

Results There were 1,930,609 patient episodes analysed and median site wait times varied from 24 to 54 minutes. Individual site model prediction median absolute errors varied from +/−22.6 minutes (95%CI 22.4,22.9) to +/− 44.0 minutes (95%CI 43.4,44.4). Global model prediction median absolute errors varied from +/−33.9 minutes (95%CI 33.4, 34.0) to +/−43.8 minutes (95%CI 43.7, 43.9). Random forest and linear regression models performed the best, rolling average models under-estimated wait times. Important variables were triage category, last-k patient average wait time, and arrival time. Wait time prediction models are not transferable across hospitals. Models performed well during the COVID-19 lockdown period.

Conclusions Electronic emergency demographic and flow information can be used to approximate emergency patient wait times. A general model is less accurate if applied without site specific factors.

What is already known on this subject

  • ⍰ Patients and families want to know approximate emergency wait times, which will improve their ability to manage their logistical, physical and emotional needs whilst waiting

  • ⍰ There are a few small studies from a limited number of jurisdictions, reporting model methods, important predictor variables and accuracy of derived models

What this study adds

  • ⍰ Our study demonstrates that predicting wait times from simple, readily available data is complex and provides estimates that aren’t as accurate as patients would like, however rough estimates may still be better than no information

  • ⍰ We present the most influential variables regarding wait times and advise against using rolling average models, preferring random forest or linear regression techniques

  • ⍰ Emergency medicine machine learning models may be less generalisable to other sites than we hope for when we read manuscripts or buy commercial off-the-shelf models or algorithms. Models developed for one site lose accuracy at another site and global models built for whole systems may need customisation to each individual site. This may apply to data science clinical decision instruments as well as operational machine learning models.

Competing Interest Statement

Some authors and collaborators are emergency physicians or directors, others work in community health (pre-hospital and district nursing). One collaborator is a consumer.

Clinical Trial

Not a clinical trial

Funding Statement

The Australian government, Medical Research Future Fund, via Monash Partners, funded this study. Researchers contributed in-kind donations of time. The Cabrini Institute and Monash University provided research infrastructure support.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The study received Monash Health ethics committee approval (RES-19-0000-763A).

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Data Availability

Data are not available for sharing. To foster future replications, we provide code snippets for model construction in an online repository.

https://doi.org/10.5281/zenodo.459978

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission.
Back to top
PreviousNext
Posted March 24, 2021.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Emergency medicine patient wait time multivariable prediction models: a multicentre derivation and validation study
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Emergency medicine patient wait time multivariable prediction models: a multicentre derivation and validation study
Katie Walker, Jirayus Jiarpakdee, Anne Loupis, Chakkrit Tantithamthavorn, Keith Joe, Michael Ben-Meir, Hamed Akhlaghi, Jennie Hutton, Wei Wang, Michael Stephenson, Gabriel Blecher, Paul Buntine, Amy Sweeny, Burak Turhan
medRxiv 2021.03.19.21253921; doi: https://doi.org/10.1101/2021.03.19.21253921
Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
Citation Tools
Emergency medicine patient wait time multivariable prediction models: a multicentre derivation and validation study
Katie Walker, Jirayus Jiarpakdee, Anne Loupis, Chakkrit Tantithamthavorn, Keith Joe, Michael Ben-Meir, Hamed Akhlaghi, Jennie Hutton, Wei Wang, Michael Stephenson, Gabriel Blecher, Paul Buntine, Amy Sweeny, Burak Turhan
medRxiv 2021.03.19.21253921; doi: https://doi.org/10.1101/2021.03.19.21253921

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Emergency Medicine
Subject Areas
All Articles
  • Addiction Medicine (76)
  • Allergy and Immunology (194)
  • Anesthesia (54)
  • Cardiovascular Medicine (488)
  • Dentistry and Oral Medicine (89)
  • Dermatology (56)
  • Emergency Medicine (168)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (211)
  • Epidemiology (5661)
  • Forensic Medicine (3)
  • Gastroenterology (215)
  • Genetic and Genomic Medicine (856)
  • Geriatric Medicine (88)
  • Health Economics (229)
  • Health Informatics (759)
  • Health Policy (388)
  • Health Systems and Quality Improvement (250)
  • Hematology (105)
  • HIV/AIDS (181)
  • Infectious Diseases (except HIV/AIDS) (6455)
  • Intensive Care and Critical Care Medicine (388)
  • Medical Education (116)
  • Medical Ethics (28)
  • Nephrology (90)
  • Neurology (845)
  • Nursing (44)
  • Nutrition (141)
  • Obstetrics and Gynecology (161)
  • Occupational and Environmental Health (258)
  • Oncology (514)
  • Ophthalmology (162)
  • Orthopedics (44)
  • Otolaryngology (105)
  • Pain Medicine (47)
  • Palliative Medicine (21)
  • Pathology (149)
  • Pediatrics (248)
  • Pharmacology and Therapeutics (146)
  • Primary Care Research (113)
  • Psychiatry and Clinical Psychology (959)
  • Public and Global Health (2222)
  • Radiology and Imaging (375)
  • Rehabilitation Medicine and Physical Therapy (174)
  • Respiratory Medicine (311)
  • Rheumatology (109)
  • Sexual and Reproductive Health (80)
  • Sports Medicine (82)
  • Surgery (118)
  • Toxicology (25)
  • Transplantation (34)
  • Urology (42)