Abstract
Symptom-Assessment Application (SAAs, e.g., NHS 111 online) that assist medical laypeople in deciding if and where to seek care (self-triage) are gaining popularity and their accuracy has been examined in numerous studies. With the public release of Large Language Models (LLMs, e.g., ChatGPT), their use in such decision-making processes is growing as well. However, there is currently no comprehensive evidence synthesis for LLMs, and no review has contextualized the accuracy of SAAs and LLMs relative to the accuracy of their users. Thus, this systematic review evaluates the self-triage accuracy of both SAAs and LLMs and compares them to the accuracy of medical laypeople. A total of 1549 studies were screened, with 19 included in the final analysis. The self-triage accuracy of SAAs was found to be moderate but highly variable (11.5 – 90.0%), while the accuracy of LLMs (57.8 – 76.0%) and laypeople (47.3 – 62.4%) was moderate with low variability. Despite some published recommendations to standardize evaluation methodologies, there remains considerable heterogeneity among studies. The use of SAAs should not be universally recommended or discouraged; rather, their utility should be assessed based on the specific use case and tool under consideration.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This study did not receive any funding.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data availability
The search strategy can be found in the Methods section and all information on studies are cited. Any additional data is available upon request.
