Abstract
Individual-based models (IBMs) informing public health policy should be calibrated to data and provide estimates of uncertainty. Two main components of model-calibration methods are the parameter-search strategy and the goodness-of-fit (GOF) measure; many options exist for each of these. This review provides an overview of the calibration methods used in IBMs modelling infectious disease spread.
We identified articles on PubMed employing simulation-based methods to calibrate IBMs in HIV, tuberculosis, and malaria epidemiology that were published between 2013 and 2018. Articles were included if models stored individual-specific information, and calibration involved comparing model output to population-level targets. We extracted information on parameter-search strategies and GOF measures. We also recorded the reporting of model validation.
The PubMed search identified 653 candidate articles, of which 84 met the criteria for review. Of the included articles, 40 (48%) combined a quantitative measure of GOF with an algorithmic parameter-search strategy, which was an optimisation algorithm (14 articles) or a sampling algorithm (26 articles). These 40 articles varied widely in their choices of parameter-search strategies and GOF measures. For the remaining 44 articles, the parameter-search strategy could either not be identified (32 articles) or was described as an informal, non-reproducible method (12 articles). Of these 44 articles, the majority (25 articles) were unclear about the GOF measure used; of the rest, only five quantitatively evaluated GOF. Thirty-two (38%) articles reported on model validation.
Less than half of the articles reviewed used algorithmic parameter-search strategies. Only one-third of articles quantified the uncertainty around calibrated parameter values and could incorporate this uncertainty into their model projections. There was no consensus on which algorithmic calibration method to use. Only 38% of the articles performed model validation. The adoption of better-documented algorithmic calibration and validation methods could improve both the reproducibility and the quality of inference in model-based epidemiology.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
WD was supported by grant 12L5816N from the Research Foundation - Flanders (FWO). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Author Declarations
All relevant ethical guidelines have been followed and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
Any clinical trials involved have been registered with an ICMJE-approved registry such as ClinicalTrials.gov and the trial ID is included in the manuscript.
Not Applicable
I have followed all appropriate research reporting guidelines and uploaded the relevant Equator, ICMJE or other checklist(s) as supplementary files, if applicable.
Not Applicable
Data Availability
Most of the data is available in the manuscript. Other data will be made available upon publication.
