Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Trade-offs between individual and ensemble forecasts of an emerging infectious disease

View ORCID ProfileRachel J. Oidtman, Elisa Omodei, Moritz U. G. Kraemer, Carlos A. Castañeda-Orjuela, Erica Cruz-Rivera, Sandra Misnaza-Castrillón, Myriam Patricia Cifuentes, Luz Emilse Rincon, Viviana Cañon, Pedro de Alarcon, Guido España, View ORCID ProfileJohn H. Huber, Sarah C. Hill, Christopher M. Barker, Michael A. Johansson, Carrie A. Manore, Robert C. Reiner Jr., Isabel Rodriguez-Barraquer, Amir S. Siraj, Enrique Frias-Martinez, Manuel García-Herranz, View ORCID ProfileT. Alex Perkins
doi: https://doi.org/10.1101/2021.02.25.21252363
Rachel J. Oidtman
1Department of Biological Sciences and Eck Institute for Global Health, University of Notre Dame, Notre Dame, IN, USA
2UNICEF, New York, NY, USA
3Department of Ecology and Evolution, University of Chicago, Chicago, IL, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Rachel J. Oidtman
  • For correspondence: roidtman@uchicago.edu mgarciaherranz@unicef.org taperkins@nd.edu
Elisa Omodei
2UNICEF, New York, NY, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Moritz U. G. Kraemer
4Department of Zoology, University of Oxford, Oxford, UK
5Boston Children’s Hospital, Boston, MA, USA
6Harvard Medical School, Boston, MA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Carlos A. Castañeda-Orjuela
7Instituto Nacional de Salud, Bogotá, Colombia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Erica Cruz-Rivera
7Instituto Nacional de Salud, Bogotá, Colombia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sandra Misnaza-Castrillón
7Instituto Nacional de Salud, Bogotá, Colombia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Myriam Patricia Cifuentes
8Ministerio de Salud y Protección Social, Bogotá, Colombia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Luz Emilse Rincon
8Ministerio de Salud y Protección Social, Bogotá, Colombia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Viviana Cañon
9UNICEF, Bogotá, Colombia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Pedro de Alarcon
10LUCA Telefonica Data Unit, Madrid, Spain
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Guido España
1Department of Biological Sciences and Eck Institute for Global Health, University of Notre Dame, Notre Dame, IN, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
John H. Huber
1Department of Biological Sciences and Eck Institute for Global Health, University of Notre Dame, Notre Dame, IN, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for John H. Huber
Sarah C. Hill
3Department of Ecology and Evolution, University of Chicago, Chicago, IL, USA
11Department of Pathobiology and Population Sciences, The Royal Veterinary College, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Christopher M. Barker
12Department of Pathology, Microbiology, and Immunology, School of Veterinary Medicince, University of California, Davis, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Michael A. Johansson
13Division of Vector-Borne Diseases, Centers for Disease Control and Prevention, San Juan, Puerto Rico
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Carrie A. Manore
14Information Systems and Modeling (A-1), Los Alamos National Laboratory, Los Alamos, NM, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Robert C. Reiner Jr.
15Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Isabel Rodriguez-Barraquer
16Department of Medicine, University of California, San Francisco, CA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Amir S. Siraj
1Department of Biological Sciences and Eck Institute for Global Health, University of Notre Dame, Notre Dame, IN, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Enrique Frias-Martinez
17Telefonica Research, Madrid, Spain
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Manuel García-Herranz
2UNICEF, New York, NY, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: roidtman@uchicago.edu mgarciaherranz@unicef.org taperkins@nd.edu
T. Alex Perkins
1Department of Biological Sciences and Eck Institute for Global Health, University of Notre Dame, Notre Dame, IN, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for T. Alex Perkins
  • For correspondence: roidtman@uchicago.edu mgarciaherranz@unicef.org taperkins@nd.edu
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

Abstract

When new pathogens emerge, numerous questions arise about their future spread, some of which can be addressed with probabilistic forecasts. The many uncertainties about the epidemiology of emerging pathogens can make it difficult to choose among model structures and assumptions, however. To assess the potential for uncertainties about emerging pathogens to affect forecasts of their spread, we evaluated the performance of a suite of 16 forecasting models in the context of the 2015-2016 Zika epidemic in Colombia. Each model featured a different combination of assumptions about the role of human mobility in driving transmission, spatiotemporal variation in transmission potential, and the number of times the virus was introduced. All models used the same core transmission model and the same iterative data assimilation algorithm to generate forecasts. By assessing forecast performance through time using logarithmic scoring with ensemble weighting, we found that which model assumptions had the most ensemble weight changed through time. In particular, spatially coupled models had higher ensemble weights in the early and late phases of the epidemic, whereas non-spatial models had higher ensemble weights at the peak of the epidemic. We compared forecast performance of the equally-weighted ensemble model to each individual model and identified a trade-off whereby certain individual models outperformed the ensemble model early in the epidemic but the ensemble model outperformed all individual models on average. On balance, our results suggest that suites of models that span uncertainty across alternative assumptions are necessary to obtain robust forecasts in the context of emerging infectious diseases.

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

RJO acknowledges support from an Eck Institute for Global Health Fellowship, GLOBES grant, Arthur J. Schmitt Fellowship, and the UNICEF Office of Innovation. MUGK is supported by The Branco Weiss Fellowship - Society in Science, administered by the ETH Zurich and acknowledges funding from the Oxford Martin School and the European Union Horizon 2020 project MOOD (\#874850). SCH is supported by the Wellcome Trust (220414/Z/20/Z).

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

No IRB approvals were necessary.

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Data Availability

The mobile phone data set used in this study is proprietary and subject to strict privacy regulations. The access to this data set was granted after reaching a non-disclosure agreement with the proprietor, who anonymized and aggregated the original data before giving access to the authors. The mobile phone is available on request after negotiation of a non-disclosure agreement with the company. The contact person is Enrique Frias-Martinez (efm{at}tid.es). Epidemiological, meteorological, and demographic data are available from Siraj et al.2018 and additionally available on https://github.com/roidtman/eid_ensemble_forecasting.

https://www.github.com/roidtman/eid_ensemble_forecasting

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license.
Back to top
PreviousNext
Posted March 01, 2021.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Trade-offs between individual and ensemble forecasts of an emerging infectious disease
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Trade-offs between individual and ensemble forecasts of an emerging infectious disease
Rachel J. Oidtman, Elisa Omodei, Moritz U. G. Kraemer, Carlos A. Castañeda-Orjuela, Erica Cruz-Rivera, Sandra Misnaza-Castrillón, Myriam Patricia Cifuentes, Luz Emilse Rincon, Viviana Cañon, Pedro de Alarcon, Guido España, John H. Huber, Sarah C. Hill, Christopher M. Barker, Michael A. Johansson, Carrie A. Manore, Robert C. Reiner Jr., Isabel Rodriguez-Barraquer, Amir S. Siraj, Enrique Frias-Martinez, Manuel García-Herranz, T. Alex Perkins
medRxiv 2021.02.25.21252363; doi: https://doi.org/10.1101/2021.02.25.21252363
Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
Citation Tools
Trade-offs between individual and ensemble forecasts of an emerging infectious disease
Rachel J. Oidtman, Elisa Omodei, Moritz U. G. Kraemer, Carlos A. Castañeda-Orjuela, Erica Cruz-Rivera, Sandra Misnaza-Castrillón, Myriam Patricia Cifuentes, Luz Emilse Rincon, Viviana Cañon, Pedro de Alarcon, Guido España, John H. Huber, Sarah C. Hill, Christopher M. Barker, Michael A. Johansson, Carrie A. Manore, Robert C. Reiner Jr., Isabel Rodriguez-Barraquer, Amir S. Siraj, Enrique Frias-Martinez, Manuel García-Herranz, T. Alex Perkins
medRxiv 2021.02.25.21252363; doi: https://doi.org/10.1101/2021.02.25.21252363

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Epidemiology
Subject Areas
All Articles
  • Addiction Medicine (216)
  • Allergy and Immunology (495)
  • Anesthesia (106)
  • Cardiovascular Medicine (1099)
  • Dentistry and Oral Medicine (196)
  • Dermatology (141)
  • Emergency Medicine (274)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (502)
  • Epidemiology (9778)
  • Forensic Medicine (5)
  • Gastroenterology (481)
  • Genetic and Genomic Medicine (2314)
  • Geriatric Medicine (223)
  • Health Economics (462)
  • Health Informatics (1563)
  • Health Policy (737)
  • Health Systems and Quality Improvement (605)
  • Hematology (238)
  • HIV/AIDS (505)
  • Infectious Diseases (except HIV/AIDS) (11653)
  • Intensive Care and Critical Care Medicine (617)
  • Medical Education (238)
  • Medical Ethics (67)
  • Nephrology (258)
  • Neurology (2146)
  • Nursing (134)
  • Nutrition (338)
  • Obstetrics and Gynecology (427)
  • Occupational and Environmental Health (518)
  • Oncology (1183)
  • Ophthalmology (365)
  • Orthopedics (128)
  • Otolaryngology (220)
  • Pain Medicine (147)
  • Palliative Medicine (50)
  • Pathology (312)
  • Pediatrics (697)
  • Pharmacology and Therapeutics (301)
  • Primary Care Research (267)
  • Psychiatry and Clinical Psychology (2186)
  • Public and Global Health (4668)
  • Radiology and Imaging (781)
  • Rehabilitation Medicine and Physical Therapy (457)
  • Respiratory Medicine (624)
  • Rheumatology (274)
  • Sexual and Reproductive Health (226)
  • Sports Medicine (210)
  • Surgery (252)
  • Toxicology (43)
  • Transplantation (120)
  • Urology (94)