Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

National and subnational short-term forecasting of COVID-19 in Germany and Poland during early 2021

View ORCID ProfileJohannes Bracher, View ORCID ProfileDaniel Wolffram, View ORCID ProfileJannik Deuschel, View ORCID ProfileKonstantin Görgen, View ORCID ProfileJakob L. Ketterer, View ORCID ProfileAlexander Ullrich, View ORCID ProfileSam Abbott, View ORCID ProfileMaria V. Barbarossa, View ORCID ProfileDimitris Bertsimas, View ORCID ProfileSangeeta Bhatia, View ORCID ProfileMarcin Bodych, View ORCID ProfileNikos I. Bosse, View ORCID ProfileJan Pablo Burgard, View ORCID ProfileLauren Castro, View ORCID ProfileGeoffrey Fairchild, Jochen Fiedler, View ORCID ProfileJan Fuhrmann, View ORCID ProfileSebastian Funk, View ORCID ProfileAnna Gambin, View ORCID ProfileKrzysztof Gogolewski, View ORCID ProfileStefan Heyder, Thomas Hotz, Yuri Kheifetz, View ORCID ProfileHolger Kirsten, View ORCID ProfileTyll Krueger, View ORCID ProfileElena Krymova, View ORCID ProfileNeele Leithäuser, Michael L. Li, View ORCID ProfileJan H. Meinke, View ORCID ProfileBłażej Miasojedow, Isaac J. Michaud, View ORCID ProfileJan Mohring, View ORCID ProfilePierre Nouvellet, Jedrzej M. Nowosielski, Tomasz Ozanski, View ORCID ProfileMaciej Radwan, Franciszek Rakowski, Markus Scholz, Saksham Soni, Ajitesh Srivastava, View ORCID ProfileTilmann Gneiting, View ORCID ProfileMelanie Schienle
doi: https://doi.org/10.1101/2021.11.05.21265810
Johannes Bracher
1Chair of Statistics and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
2Computational Statistics Group, Heidelberg Institute for Theoretical Studies (HITS), Heidelberg, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Johannes Bracher
  • For correspondence: johannes.bracher@kit.edu melanie.schienle@kit.edu
Daniel Wolffram
1Chair of Statistics and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
2Computational Statistics Group, Heidelberg Institute for Theoretical Studies (HITS), Heidelberg, Germany
3HIDSS4Health - Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Daniel Wolffram
Jannik Deuschel
1Chair of Statistics and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jannik Deuschel
Konstantin Görgen
1Chair of Statistics and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Konstantin Görgen
Jakob L. Ketterer
1Chair of Statistics and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jakob L. Ketterer
Alexander Ullrich
4Robert Koch Institute (RKI), Berlin, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Alexander Ullrich
Sam Abbott
5London School of Hygiene and Tropical Medicine, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Sam Abbott
Maria V. Barbarossa
6Frankfurt Institute for Advanced Studies, Frankfurt, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Maria V. Barbarossa
Dimitris Bertsimas
7Sloan School of Management, Massachusetts Institute of Technology, Cambridge MA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Dimitris Bertsimas
Sangeeta Bhatia
8MRC Centre for Global Infectious Disease Analysis, Abdul Latif Jameel Institute for Disease and Emergency Analytics (J-IDEA), Imperial College London, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Sangeeta Bhatia
Marcin Bodych
9Wroclaw University of Science and Technology, Wroclaw, Poland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Marcin Bodych
Nikos I. Bosse
5London School of Hygiene and Tropical Medicine, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Nikos I. Bosse
Jan Pablo Burgard
10Economic and Social Statistics Department, University of Trier, Trier, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jan Pablo Burgard
Lauren Castro
11Information Systems and Modeling, Los Alamos National Laboratory, Los Alamos, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Lauren Castro
Geoffrey Fairchild
11Information Systems and Modeling, Los Alamos National Laboratory, Los Alamos, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Geoffrey Fairchild
Jochen Fiedler
12Fraunhofer Institute for Industrial Mathematics (ITWM), Kaiserslautern, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jan Fuhrmann
13Institute for Applied Mathematics, University of Heidelberg, Heidelberg, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jan Fuhrmann
Sebastian Funk
5London School of Hygiene and Tropical Medicine, London, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Sebastian Funk
Anna Gambin
14Faculty of Mathematics, Informatics, and Mechanics, University of Warsaw, Warsaw, Poland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Anna Gambin
Krzysztof Gogolewski
14Faculty of Mathematics, Informatics, and Mechanics, University of Warsaw, Warsaw, Poland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Krzysztof Gogolewski
Stefan Heyder
15Institute of Mathematics, Technische Universität Ilmenau, Ilmenau, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Stefan Heyder
Thomas Hotz
15Institute of Mathematics, Technische Universität Ilmenau, Ilmenau, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Yuri Kheifetz
16Institute for Medical Informatics, Statistics and Epidemiology, University of Leipzig, Leipzig, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Holger Kirsten
16Institute for Medical Informatics, Statistics and Epidemiology, University of Leipzig, Leipzig, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Holger Kirsten
Tyll Krueger
9Wroclaw University of Science and Technology, Wroclaw, Poland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Tyll Krueger
Elena Krymova
17Swiss Data Science Center, ETH Zürich and EPF Lausanne, Zürich, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Elena Krymova
Neele Leithäuser
12Fraunhofer Institute for Industrial Mathematics (ITWM), Kaiserslautern, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Neele Leithäuser
Michael L. Li
18Operations Research Center, Massachusetts Institute of Technology, Cambridge, MA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jan H. Meinke
19Jülich Supercomputing Centre, Forschungszentrum Jülich, Jülich, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jan H. Meinke
Błażej Miasojedow
14Faculty of Mathematics, Informatics, and Mechanics, University of Warsaw, Warsaw, Poland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Błażej Miasojedow
Isaac J. Michaud
20Statistical Sciences Group, Los Alamos National Laboratory, Los Alamos, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jan Mohring
12Fraunhofer Institute for Industrial Mathematics (ITWM), Kaiserslautern, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jan Mohring
Pierre Nouvellet
21School of Life Sciences, University of Sussex, Brighton, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Pierre Nouvellet
Jedrzej M. Nowosielski
22Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw, Warsaw, Poland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tomasz Ozanski
9Wroclaw University of Science and Technology, Wroclaw, Poland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Maciej Radwan
22Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw, Warsaw, Poland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Maciej Radwan
Franciszek Rakowski
22Interdisciplinary Centre for Mathematical and Computational Modelling, University of Warsaw, Warsaw, Poland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Markus Scholz
16Institute for Medical Informatics, Statistics and Epidemiology, University of Leipzig, Leipzig, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Saksham Soni
18Operations Research Center, Massachusetts Institute of Technology, Cambridge, MA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ajitesh Srivastava
23Ming Hsieh Department of Computer and Electrical Engineering, University of Southern California, Los Angeles, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tilmann Gneiting
2Computational Statistics Group, Heidelberg Institute for Theoretical Studies (HITS), Heidelberg, Germany
24Institute for Stochastics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Tilmann Gneiting
Melanie Schienle
1Chair of Statistics and Econometrics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Melanie Schienle
  • For correspondence: johannes.bracher@kit.edu melanie.schienle@kit.edu
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Data/Code
  • Preview PDF
Loading

Abstract

Background During the COVID-19 pandemic there has been a strong interest in forecasts of the short-term development of epidemiological indicators to inform decision makers. In this study we evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland for the period from January through April 2021.

Methods We evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland. These were issued by 15 different forecasting models, run by independent research teams. Moreover, we study the performance of combined ensemble forecasts. Evaluation of probabilistic forecasts is based on proper scoring rules, along with interval coverage proportions to assess forecast calibration. The presented work is part of a pre-registered evaluation study and covers the period from January through April 2021.

Results We find that many, though not all, models outperform a simple baseline model up to four weeks ahead for the considered targets. Ensemble methods (i.e., combinations of different available forecasts) show very good relative performance. The addressed time period is characterized by rather stable non-pharmaceutical interventions in both countries, making short-term predictions more straightforward than in previous periods. However, major trend changes in reported cases, like the rebound in cases due to the rise of the B.1.1.7 (alpha) variant in March 2021, prove challenging to predict.

Conclusions Multi-model approaches can help to improve the performance of epidemiological forecasts. However, while death numbers can be predicted with some success based on current case and hospitalization data, predictability of case numbers remains low beyond quite short time horizons. Additional data sources including sequencing and mobility data, which were not extensively used in the present study, may help to improve performance.

Plain language summary The goal of this study is to assess the quality of forecasts of weekly case and death numbers of COVID-19 in Germany and Poland during the period of January through April 2021. We focus on real-time forecasts at time horizons of one and two weeks ahead created by fourteen independent teams. Forecasts are systematically evaluated taking uncertainty ranges of predictions into account. We find that combining different forecasts into ensembles can improve the quality of predictions, but especially case numbers proved very challenging to predict beyond quite short time windows. Additional data sources, in particular genetic sequencing data, may help to improve forecasts in the future.

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

J. Bracher, M. Schienle and T. Gneiting acknowledge support from the Helmholtz Foundation via the SIMCARD Information and Data Science Pilot Project. T. Gneiting and D. Wolffram are grateful for support by the Klaus Tschira Foundation. D. Wolffram's contribution was moreover supported by the Helmholtz Association under the joint research school HIDSS4Health - Helmholtz Information and Data Science School for Health. N.I. Bosse was supported by the Health Protection Research Unit (grant code NIHR200908). S. Funk and S. Abbott were supperted by the Wellcome Trust (210758/Z/18/Z). The itwm-dSEIR forecasting team (J. Fiedler, N. Leith\"auser, J. Mohring) was supported by the Ministry of Health and Science of Rhineland Palatinate and the Fraunhofer Anti-Corona Program. S. Bhatia acknowledges funding from the Wellcome Trust (219415). Work on the ICM UW epidemiological model (J.M Nowosielski, M. Radwan, F. Rakowski) was supported by the Polish Minister of Science and Higher Education grant 51/WFSN/2020 given to the University of Warsaw. Development of the IMISE-SECIR model (Y. Kheifetz, H. Kirsten, S. Scholz) was funded in the framework of the project SaxoCOV (Saxonian COVID-19 Research Consortium). SaxoCOV was co-financed with tax funds on the basis of the budget passed by the Saxon state parliament. Model presentation was funded by the NFDI4Health Task Force COVID-19 (www.nfdi4health.de/task-force-covid-19-2) within DFG project LO-342/17-1.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

This study contains publicly available surveillance data on COVID-19 (from Robert Koch Institute, the Polish Ministry of Health and Johns Hopkins CSSE). These data used have been deposited at https://github.com/KITmetricslab/covid19-forecast-hub-de

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Footnotes

  • This version has a slightly different structure and addresses a few weaknesses of the previous version (e.g., by including a discussion on case ascertainment rates). The main messages of the paper remain unchanged. The title has been slightly changed and a plain language summary has been included. Three additional authors (Castro, Fairchild, Michaud) have been included who due to clearance questions had been removed from the first version.

Data Availability

All data produced are available at https://github.com/KITmetricslab/covid19-forecast-hub-de

https://github.com/KITmetricslab/covid19-forecast-hub-de

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC 4.0 International license.
Back to top
PreviousNext
Posted August 26, 2022.
Download PDF
Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
National and subnational short-term forecasting of COVID-19 in Germany and Poland during early 2021
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
National and subnational short-term forecasting of COVID-19 in Germany and Poland during early 2021
Johannes Bracher, Daniel Wolffram, Jannik Deuschel, Konstantin Görgen, Jakob L. Ketterer, Alexander Ullrich, Sam Abbott, Maria V. Barbarossa, Dimitris Bertsimas, Sangeeta Bhatia, Marcin Bodych, Nikos I. Bosse, Jan Pablo Burgard, Lauren Castro, Geoffrey Fairchild, Jochen Fiedler, Jan Fuhrmann, Sebastian Funk, Anna Gambin, Krzysztof Gogolewski, Stefan Heyder, Thomas Hotz, Yuri Kheifetz, Holger Kirsten, Tyll Krueger, Elena Krymova, Neele Leithäuser, Michael L. Li, Jan H. Meinke, Błażej Miasojedow, Isaac J. Michaud, Jan Mohring, Pierre Nouvellet, Jedrzej M. Nowosielski, Tomasz Ozanski, Maciej Radwan, Franciszek Rakowski, Markus Scholz, Saksham Soni, Ajitesh Srivastava, Tilmann Gneiting, Melanie Schienle
medRxiv 2021.11.05.21265810; doi: https://doi.org/10.1101/2021.11.05.21265810
Digg logo Reddit logo Twitter logo Facebook logo Google logo LinkedIn logo Mendeley logo
Citation Tools
National and subnational short-term forecasting of COVID-19 in Germany and Poland during early 2021
Johannes Bracher, Daniel Wolffram, Jannik Deuschel, Konstantin Görgen, Jakob L. Ketterer, Alexander Ullrich, Sam Abbott, Maria V. Barbarossa, Dimitris Bertsimas, Sangeeta Bhatia, Marcin Bodych, Nikos I. Bosse, Jan Pablo Burgard, Lauren Castro, Geoffrey Fairchild, Jochen Fiedler, Jan Fuhrmann, Sebastian Funk, Anna Gambin, Krzysztof Gogolewski, Stefan Heyder, Thomas Hotz, Yuri Kheifetz, Holger Kirsten, Tyll Krueger, Elena Krymova, Neele Leithäuser, Michael L. Li, Jan H. Meinke, Błażej Miasojedow, Isaac J. Michaud, Jan Mohring, Pierre Nouvellet, Jedrzej M. Nowosielski, Tomasz Ozanski, Maciej Radwan, Franciszek Rakowski, Markus Scholz, Saksham Soni, Ajitesh Srivastava, Tilmann Gneiting, Melanie Schienle
medRxiv 2021.11.05.21265810; doi: https://doi.org/10.1101/2021.11.05.21265810

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Epidemiology
Subject Areas
All Articles
  • Addiction Medicine (214)
  • Allergy and Immunology (495)
  • Anesthesia (106)
  • Cardiovascular Medicine (1091)
  • Dentistry and Oral Medicine (194)
  • Dermatology (141)
  • Emergency Medicine (274)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (497)
  • Epidemiology (9747)
  • Forensic Medicine (5)
  • Gastroenterology (480)
  • Genetic and Genomic Medicine (2298)
  • Geriatric Medicine (221)
  • Health Economics (461)
  • Health Informatics (1548)
  • Health Policy (729)
  • Health Systems and Quality Improvement (600)
  • Hematology (236)
  • HIV/AIDS (500)
  • Infectious Diseases (except HIV/AIDS) (11620)
  • Intensive Care and Critical Care Medicine (615)
  • Medical Education (236)
  • Medical Ethics (67)
  • Nephrology (256)
  • Neurology (2136)
  • Nursing (133)
  • Nutrition (332)
  • Obstetrics and Gynecology (424)
  • Occupational and Environmental Health (516)
  • Oncology (1171)
  • Ophthalmology (363)
  • Orthopedics (128)
  • Otolaryngology (220)
  • Pain Medicine (145)
  • Palliative Medicine (50)
  • Pathology (308)
  • Pediatrics (692)
  • Pharmacology and Therapeutics (298)
  • Primary Care Research (265)
  • Psychiatry and Clinical Psychology (2168)
  • Public and Global Health (4640)
  • Radiology and Imaging (775)
  • Rehabilitation Medicine and Physical Therapy (450)
  • Respiratory Medicine (622)
  • Rheumatology (273)
  • Sexual and Reproductive Health (224)
  • Sports Medicine (208)
  • Surgery (250)
  • Toxicology (42)
  • Transplantation (120)
  • Urology (94)