Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)

View ORCID ProfileSuhana Bedi, View ORCID ProfileYutong Liu, View ORCID ProfileLucy Orr-Ewing, View ORCID ProfileDev Dash, View ORCID ProfileSanmi Koyejo, View ORCID ProfileAlison Callahan, View ORCID ProfileJason A. Fries, View ORCID ProfileMichael Wornow, View ORCID ProfileAkshay Swaminathan, View ORCID ProfileLisa Soleymani Lehmann, Hyo Jung Hong, View ORCID ProfileMehr Kashyap, Akash R. Chaurasia, Nirav R. Shah, Karandeep Singh, Troy Tazbaz, View ORCID ProfileArnold Milstein, View ORCID ProfileMichael A. Pfeffer, View ORCID ProfileNigam H. Shah
doi: https://doi.org/10.1101/2024.04.15.24305869
Suhana Bedi
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Suhana Bedi
Yutong Liu
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Yutong Liu
Lucy Orr-Ewing
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Lucy Orr-Ewing
Dev Dash
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Dev Dash
Sanmi Koyejo
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Sanmi Koyejo
Alison Callahan
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Alison Callahan
Jason A. Fries
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jason A. Fries
Michael Wornow
2Stanford
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Michael Wornow
Akshay Swaminathan
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Akshay Swaminathan
Lisa Soleymani Lehmann
3Harvard University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Lisa Soleymani Lehmann
Hyo Jung Hong
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Mehr Kashyap
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Mehr Kashyap
Akash R. Chaurasia
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nirav R. Shah
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Karandeep Singh
4University of California San Diego
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Troy Tazbaz
5US Food and Drug Administration
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Arnold Milstein
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Arnold Milstein
Michael A. Pfeffer
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Michael A. Pfeffer
Nigam H. Shah
1Stanford University
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Nigam H. Shah
  • For correspondence: nigam{at}stanford.edu
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

Article Information

doi 
https://doi.org/10.1101/2024.04.15.24305869
History 
  • April 18, 2024.

Article Versions

  • Version 1 (April 16, 2024 - 20:54).
  • You are currently viewing Version 2 of this article (April 18, 2024 - 12:06).
  • Version 3 (May 7, 2024 - 12:07).
  • View Version 4, the most recent version of this article.
Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC-ND 4.0 International license.

Author Information

  1. Suhana Bedi1,
  2. Yutong Liu1,
  3. Lucy Orr-Ewing1,
  4. Dev Dash1,
  5. Sanmi Koyejo1,
  6. Alison Callahan1,
  7. Jason A. Fries1,
  8. Michael Wornow2,
  9. Akshay Swaminathan1,
  10. Lisa Soleymani Lehmann3,
  11. Hyo Jung Hong1,
  12. Mehr Kashyap1,
  13. Akash R. Chaurasia1,
  14. Nirav R. Shah1,
  15. Karandeep Singh4,
  16. Troy Tazbaz5,
  17. Arnold Milstein1,
  18. Michael A. Pfeffer1 and
  19. Nigam H. Shah1,*
  1. 1Stanford University
  2. 2Stanford
  3. 3Harvard University
  4. 4University of California San Diego
  5. 5US Food and Drug Administration
  1. ↵*Corresponding author; email: nigam{at}stanford.edu
Back to top
PreviousNext
Posted April 18, 2024.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)
Suhana Bedi, Yutong Liu, Lucy Orr-Ewing, Dev Dash, Sanmi Koyejo, Alison Callahan, Jason A. Fries, Michael Wornow, Akshay Swaminathan, Lisa Soleymani Lehmann, Hyo Jung Hong, Mehr Kashyap, Akash R. Chaurasia, Nirav R. Shah, Karandeep Singh, Troy Tazbaz, Arnold Milstein, Michael A. Pfeffer, Nigam H. Shah
medRxiv 2024.04.15.24305869; doi: https://doi.org/10.1101/2024.04.15.24305869
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
A Systematic Review of Testing and Evaluation of Healthcare Applications of Large Language Models (LLMs)
Suhana Bedi, Yutong Liu, Lucy Orr-Ewing, Dev Dash, Sanmi Koyejo, Alison Callahan, Jason A. Fries, Michael Wornow, Akshay Swaminathan, Lisa Soleymani Lehmann, Hyo Jung Hong, Mehr Kashyap, Akash R. Chaurasia, Nirav R. Shah, Karandeep Singh, Troy Tazbaz, Arnold Milstein, Michael A. Pfeffer, Nigam H. Shah
medRxiv 2024.04.15.24305869; doi: https://doi.org/10.1101/2024.04.15.24305869

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Health Informatics
Subject Areas
All Articles
  • Addiction Medicine (576)
  • Allergy and Immunology (867)
  • Anesthesia (306)
  • Cardiovascular Medicine (4480)
  • Dentistry and Oral Medicine (449)
  • Dermatology (385)
  • Emergency Medicine (614)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (1528)
  • Epidemiology (15276)
  • Forensic Medicine (31)
  • Gastroenterology (1133)
  • Genetic and Genomic Medicine (6643)
  • Geriatric Medicine (671)
  • Health Economics (1006)
  • Health Informatics (4602)
  • Health Policy (1378)
  • Health Systems and Quality Improvement (1622)
  • Hematology (544)
  • HIV/AIDS (1275)
  • Infectious Diseases (except HIV/AIDS) (15959)
  • Intensive Care and Critical Care Medicine (1110)
  • Medical Education (626)
  • Medical Ethics (147)
  • Nephrology (674)
  • Neurology (6692)
  • Nursing (346)
  • Nutrition (1006)
  • Obstetrics and Gynecology (1152)
  • Occupational and Environmental Health (961)
  • Oncology (3369)
  • Ophthalmology (988)
  • Orthopedics (370)
  • Otolaryngology (421)
  • Pain Medicine (437)
  • Palliative Medicine (131)
  • Pathology (668)
  • Pediatrics (1703)
  • Pharmacology and Therapeutics (699)
  • Primary Care Research (717)
  • Psychiatry and Clinical Psychology (5494)
  • Public and Global Health (9284)
  • Radiology and Imaging (2223)
  • Rehabilitation Medicine and Physical Therapy (1375)
  • Respiratory Medicine (1201)
  • Rheumatology (598)
  • Sexual and Reproductive Health (720)
  • Sports Medicine (535)
  • Surgery (720)
  • Toxicology (100)
  • Transplantation (290)
  • Urology (266)