RT Journal Article
SR Electronic
T1 A Quantitative Evaluation of COVID-19 Epidemiological Models
JF medRxiv
FD Cold Spring Harbor Laboratory Press
SP 2021.02.06.21251276
DO 10.1101/2021.02.06.21251276
A1 Osman N. Yogurtcu
A1 Marisabel Rodriguez Messan
A1 Richard C. Gerkin
A1 Artur A. Belov
A1 Hong Yang
A1 Richard A Forshee
A1 Carson C. Chow
YR 2021
UL http://medrxiv.org/content/early/2021/02/08/2021.02.06.21251276.abstract
AB Quantifying how accurate epidemiological models of COVID-19 forecast the number of future cases and deaths can help frame how to incorporate mathematical models to inform public health decisions. Here we analyze and score the predictive ability of publicly available COVID-19 epidemiological models on the COVID-19 Forecast Hub. Our score uses the posted forecast cumulative distributions to compute the log-likelihood for held-out COVID-19 positive cases and deaths. Scores are updated continuously as new data become available, and model performance is tracked over time. We use model scores to construct ensemble models based on past performance. Our publicly available quantitative framework may aid in improving modeling frameworks, and assist policy makers in selecting modeling paradigms to balance the delicate trade-offs between the economy and public health.Competing Interest StatementThe authors have declared no competing interest.Funding StatementMRM was supported in part by an appointment to the Research Participation Program at OBE/CBER, U.S. Food and Drug Administration, administered by the Oak Ridge Institute for Science and Education through an interagency agreement between the U.S. Department of Energy and FDA. CCC is supported by the Intramural Research Program of the NIH/NIDDK. The authors received no financial support from any source, and there is no conflict of interest. This article reflects the views of the authors and should not be construed to represent FDA's views or policies.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:All relevant ethical guidelines have been followed; any necessary IRB and/or ethics committee approvals have been obtained.All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesAll data and computer code used is publicly available https://github.com/ONYLAB/Scoreboard