Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review

William Wallace, Calvin Chan, Swathikan Chidambaram, Lydia Hanna, Fahad Mujtaba Iqbal, Amish Acharya, Pasha Normahani, View ORCID ProfileHutan Ashrafian, Sheraz R Markar, Viknesh Sounderajah, Ara Darzi
doi: https://doi.org/10.1101/2021.12.21.21268167
William Wallace
1Department of Surgery & Cancer, Imperial College London, St. Mary’s Hospital, London, W2 1NY, UK
Roles: Medical Student
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Calvin Chan
1Department of Surgery & Cancer, Imperial College London, St. Mary’s Hospital, London, W2 1NY, UK
Roles: Medical Student
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Swathikan Chidambaram
1Department of Surgery & Cancer, Imperial College London, St. Mary’s Hospital, London, W2 1NY, UK
Roles: Academic Clinical Fellow
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lydia Hanna
1Department of Surgery & Cancer, Imperial College London, St. Mary’s Hospital, London, W2 1NY, UK
Roles: Specialist Registrar
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Fahad Mujtaba Iqbal
1Department of Surgery & Cancer, Imperial College London, St. Mary’s Hospital, London, W2 1NY, UK
2Institute of Global Health Innovation, Imperial College London, South Kensington Campus, London, SW7 2AZ, UK
Roles: Clinical Research Fellow
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Amish Acharya
1Department of Surgery & Cancer, Imperial College London, St. Mary’s Hospital, London, W2 1NY, UK
2Institute of Global Health Innovation, Imperial College London, South Kensington Campus, London, SW7 2AZ, UK
Roles: Clinical Research Fellow
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Pasha Normahani
1Department of Surgery & Cancer, Imperial College London, St. Mary’s Hospital, London, W2 1NY, UK
Roles: NIHR Academic Clinical Lecturer
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hutan Ashrafian
1Department of Surgery & Cancer, Imperial College London, St. Mary’s Hospital, London, W2 1NY, UK
2Institute of Global Health Innovation, Imperial College London, South Kensington Campus, London, SW7 2AZ, UK
Roles: Honorary Senior Research Fellow
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Hutan Ashrafian
Sheraz R Markar
3Department of Molecular Medicine and Surgery, Karolinska Institutet, Stockholm, Sweden
4Nuffield Department of Surgery, Churchill Hospital, University of Oxford, OX3 7LE, UK
Roles: Assistant Professor
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Viknesh Sounderajah
1Department of Surgery & Cancer, Imperial College London, St. Mary’s Hospital, London, W2 1NY, UK
2Institute of Global Health Innovation, Imperial College London, South Kensington Campus, London, SW7 2AZ, UK
Roles: Clinical Research Fellow
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: vs1108@imperial.ac.uk
Ara Darzi
1Department of Surgery & Cancer, Imperial College London, St. Mary’s Hospital, London, W2 1NY, UK
2Institute of Global Health Innovation, Imperial College London, South Kensington Campus, London, SW7 2AZ, UK
Roles: Professor of Surgery
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

ABSTRACT

Objective To evaluate the accuracy of digital and online symptom checkers in providing diagnoses and appropriate triage advice.

Design Systematic review.

Data sources Medline and Web of Science were searched up to 15 February 2021.

Eligibility criteria for study selection Prospective and retrospective cohort, vignette, or audit studies that utilised an online or application-based service designed to input symptoms and biodata in order to generate diagnoses, health advice and direct patients to appropriate services were included.

Main outcome measures The primary outcomes were (1) the accuracy of symptom checkers for providing the correct diagnosis and (2) the accuracy of subsequent triage advice given.

Data extraction and synthesis Data extraction and quality assessment (using the QUADAS-2 tool) were performed by two independent reviewers. Owing to heterogeneity of the studies, meta-analysis was not possible. A narrative synthesis of the included studies and pre-specified outcomes was completed.

Results Of the 177 studies retrieved, nine cohort studies and one cross-sectional study met the inclusion criteria. Symptom checkers evaluated a variety of medical conditions including ophthalmological conditions, inflammatory arthritides and HIV. 50% of the studies recruited real patients, while the remainder used simulated cases. The diagnostic accuracy of the primary diagnosis was low (range: 19% to 36%) and varied between individual symptom checkers, despite consistent symptom data input. Triage accuracy (range: 48.8% to 90.1%) was typically higher than diagnostic accuracy. Of note, one study found that 78.6% of emergency ophthalmic cases were under-triaged.

Conclusions The diagnostic and triage accuracy of symptom checkers are variable and of low accuracy. Given the increasing push towards population-wide digital health technology adoption, reliance upon symptom checkers in lieu of traditional assessment models, poses the potential for clinical risk. Further primary studies, utilising improved study reporting, core outcome sets and subgroup analyses, are warranted to demonstrate equitable and non-inferior performance of these technologies to that of current best practice.

PROSPERO registration number CRD42021271022.

What is already known on this topic Chambers et al. (2019) have previously examined the evidence underpinning digital and online symptom checkers, including the accuracy of the diagnostic and triage information, for urgent health problems and found that diagnostic accuracy was generally low and varied depending on the symptom checker used. Given the increased reliance upon digital health technologies by health systems in light of the ongoing COVID-19 pandemic, in addition to the marked increase in availability of similarly themed digital health products since the last systematic review, a contemporary and comprehensive reassessment of this class of technologies to ascertain their diagnostic and triage accuracy is warranted.

What this study adds Our systematic review demonstrates that the diagnostic accuracy of symptom checkers remains low and varies significantly depending on the pathology or symptom checker used.

The findings of this systematic review suggests that this class of technologies, in their current state, poses significant risk for patient safety, particularly if utilised in isolation.

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

Infrastructure support for this research was provided by the NIHR Imperial Biomedical Research Centre (BRC).

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Footnotes

  • * Joint first authorship

Data Availability

All data produced in the present work are contained in the manuscript

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license.
Back to top
PreviousNext
Posted December 21, 2021.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review
William Wallace, Calvin Chan, Swathikan Chidambaram, Lydia Hanna, Fahad Mujtaba Iqbal, Amish Acharya, Pasha Normahani, Hutan Ashrafian, Sheraz R Markar, Viknesh Sounderajah, Ara Darzi
medRxiv 2021.12.21.21268167; doi: https://doi.org/10.1101/2021.12.21.21268167
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review
William Wallace, Calvin Chan, Swathikan Chidambaram, Lydia Hanna, Fahad Mujtaba Iqbal, Amish Acharya, Pasha Normahani, Hutan Ashrafian, Sheraz R Markar, Viknesh Sounderajah, Ara Darzi
medRxiv 2021.12.21.21268167; doi: https://doi.org/10.1101/2021.12.21.21268167

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Public and Global Health
Subject Areas
All Articles
  • Addiction Medicine (269)
  • Allergy and Immunology (549)
  • Anesthesia (134)
  • Cardiovascular Medicine (1747)
  • Dentistry and Oral Medicine (238)
  • Dermatology (172)
  • Emergency Medicine (310)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (653)
  • Epidemiology (10780)
  • Forensic Medicine (8)
  • Gastroenterology (584)
  • Genetic and Genomic Medicine (2933)
  • Geriatric Medicine (286)
  • Health Economics (531)
  • Health Informatics (1918)
  • Health Policy (833)
  • Health Systems and Quality Improvement (743)
  • Hematology (290)
  • HIV/AIDS (627)
  • Infectious Diseases (except HIV/AIDS) (12496)
  • Intensive Care and Critical Care Medicine (684)
  • Medical Education (299)
  • Medical Ethics (86)
  • Nephrology (321)
  • Neurology (2780)
  • Nursing (150)
  • Nutrition (431)
  • Obstetrics and Gynecology (554)
  • Occupational and Environmental Health (597)
  • Oncology (1454)
  • Ophthalmology (440)
  • Orthopedics (172)
  • Otolaryngology (255)
  • Pain Medicine (190)
  • Palliative Medicine (56)
  • Pathology (379)
  • Pediatrics (865)
  • Pharmacology and Therapeutics (362)
  • Primary Care Research (333)
  • Psychiatry and Clinical Psychology (2630)
  • Public and Global Health (5338)
  • Radiology and Imaging (1002)
  • Rehabilitation Medicine and Physical Therapy (594)
  • Respiratory Medicine (722)
  • Rheumatology (329)
  • Sexual and Reproductive Health (288)
  • Sports Medicine (278)
  • Surgery (327)
  • Toxicology (47)
  • Transplantation (149)
  • Urology (125)