RT Journal Article
SR Electronic
T1 A benchmark of online COVID-19 symptom checkers
JF medRxiv
FD Cold Spring Harbor Laboratory Press
SP 2020.05.22.20109777
DO 10.1101/2020.05.22.20109777
A1 Munsch, Nicolas
A1 Martin, Alistair
A1 Gruarin, Stefanie
A1 Nateqi, Jama
A1 Abdarahmane, Isselmou
A1 Weingartner-Ortner, Rafael
A1 Knapp, Bernhard
YR 2020
UL http://medrxiv.org/content/early/2020/08/19/2020.05.22.20109777.abstract
AB Background A large number of online COVID-19 symptom checkers and chatbots have been developed but anecdotal evidence suggests that their conclusions are highly variable. To our knowledge, no study has evaluated the accuracy of COVID-19 symptom checkers in a statistically rigorous manner.Methods In this paper, we evaluate 10 different COVID-19 symptom checkers screening 50 COVID-19 case reports alongside 410 non-COVID-19 control cases.Results We find that the number of correctly assessed cases varies considerably between different symptom checkers, with Symptoma (F1=0.92, MCC=0.85) showing the overall best performance followed by Infermedica (F1=0.80, MCC=0.61).Competing Interest StatementAll authors are employees of Symptoma GmbH. JN holds shares of Symptoma.Funding StatementThis study has received funding from the European Unions Horizon 2020 research and innovation programme under grant agreement No 830017Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:pure computer work on public dataAll necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesAll analysis can be reproduced via the web-interfaces given in Table 1