Fundamental limits on inferring epidemic resurgence in real time using effective reproduction numbers

Kris V. Parag, Christl A. Donnelly
doi: https://doi.org/10.1101/2021.09.08.21263270

Abstract

We find that epidemic resurgence, defined as an upswing in the effective reproduction number (R) of the contagion from subcritical to supercritical values, is fundamentally difficult to detect in real time. Inherent latencies in pathogen transmission, coupled with smaller and intrinsically noisier case incidence across periods of subcritical spread, mean that resurgence cannot be reliably detected without significant delays of the order of the generation time of the disease, even when case reporting is perfect. In contrast, epidemic suppression (where R falls from supercritical to subcritical values) may be ascertained 5–10 times faster due to the naturally larger incidence at which control actions are generally applied. We prove that these innate limits on detecting resurgence only worsen when spatial or demographic heterogeneities are incorporated. Consequently, we argue that resurgence is more effectively handled proactively, potentially at the expense of false alarms. Timely responses to recrudescent infections or emerging variants of concern are more likely to be possible when policy is informed by a greater quality and diversity of surveillance data than by further optimisation of the statistical models used to process routine outbreak data.

Author summary The timely detection of epidemic resurgence (i.e., upcoming waves of infected cases) is crucial for informing public health policy, providing valuable signals for implementing interventions and identifying emerging pathogenic variants or important population-level behavioural shifts. Increases in epidemic transmissibility, parametrised by the time-varying reproduction number, R, commonly signify resurgence. While many studies have improved computational methods for inferring R from case data, to enhance real-time resurgence detection, few have examined what limits, if any, fundamentally restrict our ability to perform this inference. We apply optimal Bayesian detection algorithms and sensitivity tests and discover that resurgent (upward) R-changes are intrinsically more difficult to detect than equivalent downward changes indicating control. This asymmetry derives from the often lower and stochastically noisier case numbers that associate with resurgence, and induces detection delays on the order of the disease generation time. We prove these delays only worsen if spatial or demographic differences in transmissibility are modelled. As these fundamental limits exist even if case data are perfect, we conclude that designing integrated surveillance systems that fuse potentially timelier data sources (e.g., wastewater) may be more important than improving R-estimation methodology and deduce that there may be merit (subject to false alarm costs) in conservative resurgence response initiatives.

Introduction

Real-time estimates of the transmissibility of an infectious disease [1,2] are crucial for informed outbreak responses. Timely detection of salient changes in the effective reproduction number (R) of the disease of interest, which measures the average number of secondary cases likely caused by a typical primary case, can provide important evidence for policymaking and public communication [3,4], as well as improve forecasts of disease burden [5] (e.g., hospitalisations and deaths). Two critical changes of interest are resurgence and control. Resurgence, which we define as an increase from subcritical (R ≤ 1) to supercritical (R > 1) transmissibility, can warn of imminent waves of infections, signify the emergence of pathogenic variants of concern and signal important shifts in the behavioural patterns of population [6,7]. Alternatively, control (or suppression) describes a switch from supercritical to subcritical spread and can indicate the effectiveness of interventions and the impact of depleting susceptibility (including that due to vaccine-induced immunity) [8,9].

Identifying these transmissibility changes in real time, however, is an enduring challenge for statistical modelling and surveillance planning. Inferring a transition in R from stochastic time series of incident cases necessitates assumptions about the differences among meaningful variations (signal) and random fluctuations (noise) [10–12]. Modern approaches to epidemic modelling and monitoring aim to maximise this signal-to-noise ratio either by enhancing noise filtering and bias correction methods [13–15], or by amplifying signal fidelity through improving surveillance quality and diversity [16–18]. While both approaches have substantially advanced the field, there have been few attempts to explore what, if any, fundamental limits exist on the timely detection of these changes. Such limits can provide key benchmarks for assessing the effectiveness of modelling or data collection and deepen our understanding of what can and cannot be achieved by real-time outbreak response programmes, ensuring that model outputs are not overinterpreted and redirecting surveillance resources more efficiently [19–21].

While studies are examining intrinsic bounds on epidemic monitoring and forecasting [22–25], works on transmissibility have mostly probed how extrinsic surveillance biases might cause R misestimation [14,26–28]. Here we address these gaps in the literature by characterising and exposing fundamental limits to detecting resurgence and control, from a perfectly ascertained incidence time series, using effective reproduction numbers. This presents vital insights into the best real-time performance possible and blueprints for how outbreak preparedness might be improved. We analyse a predominant, flexible real-time epidemic model [1,2] and discover stark asymmetries in our intrinsic ability to detect resurgence and control, emerging from the noisier, low-incidence data underlying possible resurgence events. While epidemic control or suppression change-points are inferred robustly and rapidly, the data bottleneck caused by subcritical spread forces inherent delays (potentially 5–10 times that for control and on the order of the mean disease generation time) that inhibit real-time resurgence estimation.

We show that these innate constraints on resurgence detection worsen with smaller epidemic size, steepness of the upswing in R and spatial or demographic heterogeneities. Given these limitations to timely outbreak analysis, which exist despite perfect case reporting and the use of optimal Bayesian detection algorithms [15,29], we argue that methodological improvements to existing models for analysing epidemic curves (e.g., cases, hospitalisations or deaths) are less important than designing enhanced and integrated surveillance systems [30,31]. Such systems, which might fuse multiple data streams including novel ones (e.g., wastewater [32]) to triangulate possible resurgences, could minimize some of these fundamental bottlenecks. We conclude that early responses to suspected resurging epidemics, at the expense of false alarms, might be justified in many settings, both from our analysis and the consensus that lags in implementing interventions can translate into severely elevated epidemic burden [33–36]. While such decisions must, ultimately, be weighed against the cost of those interventions, the bottlenecks we expose, hopefully, bolster the evidence base for decision-making. Using theory and simulation, we explore and elucidate these conclusions in the next section.

Results

Epidemic resurgence is statistically more difficult to infer than control

We first provide intuition for why resurgence and control might present asymmetric difficulties when inferring transmissibility in real time. We consider an epidemic modelled via a renewal branching process [37] over times (usually in days) 1 ≤ s ≤ t. Such models have been widely applied to infer the transmissibility of many diseases including COVID-19, pandemic influenza and Ebola virus disease. Renewal models postulate that the incidence of new cases at time s, denoted Is, depends on the effective reproduction number, Rs, and the past incidence, Embedded Image as in Eq. (1) [2]. Here Embedded Image means the set {Ia, Ia+1, …, Ib} and ≡ indicates equality in distribution. Embedded Image In Eq. (1), Pois represents Poisson noise and Λs is the total infectiousness, which summarises the weighted influence of past infections. The set of weights wu for all u define the generation time distribution of the infectious disease with Embedded Image [38]. We assume that all wu are known. If this distribution changes across the epidemic [39], we can recompute the Λs terms after that change to model its effects. Applying Bayesian inference techniques (see Methods in the S1 Appendix for derivations) [2,40] under the assumption that transmissibility is constant over a past window of size m days, τ(s) = {s, s − 1, …, s − m + 1}, we obtain the gamma (Gam) posterior distribution given the incidence data Embedded Image, with sums of iτ(s) = ∑u∈τ(s) Iu and λτ(s) = ∑u∈τ(s) Λu.

Here (a, c) are prior distribution (P(Rs)) parameters, which are set so the prior mean of Rs is above 1 but uninformative. This maximises sensitivity to resurgence since the model, in the absence of data, favours E[Rs] > 1. The approximations above and later emerge from the window assumption and underpin popular real-time R-inference methods [2,41]. Using this renewal formulation, we define the relative change in the epidemic size as Embedded Image. This measures the perturbation to the past incidence (summarised by λτ(s)) that the most recently observed incidence, iτ(s), causes over τ(s). Normalising by λτ(s) is sensible as the posterior mean estimate of Rs is roughly Embedded Image, so Δλτ(s) approximates Rs − 1.

This posterior distribution only uses data up until time s and defines our real-time estimate of R at that time. We can analyse its properties (and related likelihood function Embedded Image to obtain the Fisher information (FI) on the left side of Eq. (2). We derive this expression in the Methods of the S1 Appendix. This FI captures how informative Embedded Image is (here approximated by Embedded Image) for inferring Rs, with its inverse defining the smallest asymptotic variance of any Rs estimate [10,42]. Larger FI implies better statistical precision. Embedded Image As resurgence will likely follow low-incidence periods, we might expect λτ(s) to be small, while Rs rises. This effect will reduce the FI in Eq. (2), making these changes harder to detect. In contrast, the impact of interventions will be easier to infer since these are often applied when cases are larger and reduce Rs. This observation applies for any τ(s) and is fundamental as it delimits the best estimator performance under our renewal model (Cramer-Rao bound) [43].

We expand on this intuition, using the R posterior distribution to derive (see S1 Appendix Methods) the real-time resurgence probability Embedded Image, as on the right side of Eq. (2). We plot its implications in Fig 1, corroborating our intuition. In panel A we find that larger past epidemic sizes (λτ(s)) improve our ability to detect transmissibility shifts from fluctuations in incidence (the posterior distributions for Rs overlap less). Panel B bolsters this idea, showing that when λτ(s) is smaller (as is likely before resurgence) we need to observe larger relative epidemic size changes (Δλτ(s)) for some increase in Embedded Image than for an equivalent decrease when aiming to detect control (where λτ(s) will often be larger). This detection asymmetry holds for arbitrary window sizes and indicates that data bottlenecks translate into real-time detection delays. We assess the magnitude of these delays next.

Fig 1:
Fig 1: Relative sensitivity to perturbations in incidence.

Panel A plots posterior real-time distributions for time-varying reproduction numbers Rs, given incidence data Embedded Image, at different relative incidence perturbations, Embedded Image, (increasing from blue to red). Here τ(s) represents some arbitrary window size used in computation (see Eq. (2)). The degree of distribution separation and hence our ability to uncover meaningful incidence fluctuations from noise, improves with the current epidemic size, λτ(s) (i.e., as this increases from 25–400 overlap among the distributions decreases). Panel B shows how this sensitivity modulates our capacity to infer resurgence Embedded Image and control Embedded Image. If epidemic size is smaller, larger relative incidence perturbations are required to detect the same change in Rs (curves have gentler gradient as we traverse from blue to red). Resurgence (likely closer to the blue line in the top right quadrant) is appreciably and innately harder to detect than control (likely closer to the red line, in the bottom left quadrant).

Fundamental delays on detecting resurgence but not control

The intrinsic asymmetry in sensitivity to upward versus downward shifts in R (see Fig 1) implies that it is not equally simple to infer resurgence and control from incident cases. We investigate ramifications of this observation by comparing our real-time Rs-estimates to ones exploiting all the future incidence information available. We no longer consider window-based approximations (which we only use to extract analytic insights) but instead apply formal real-time Bayesian inference and detection algorithms [29]. We investigate two foundational posterior distributions, the filtered, ps, and smoothed, qs, distributions, defined as in Eq. (3). Embedded Image Here ps considers all information until time s and captures changes in Rs from Embedded Image in real time. Estimates of Rs using this posterior distribution minimise the mean squared error (MSE) given Embedded Image. In contrast, qs extracts all the information from the full incidence curve Embedded Image, providing the minimum MSE Rs-estimate given Embedded Image [29]. This smoother MSE is never larger and may be substantially smaller than the filtered MSE due to its use of additional information (i.e., Embedded Image) [29,44]. The differential between ps and qs, summarised via the Kullback-Liebler divergence, D(ps|qs), measures the value of this additional ‘future’ information.

Bayesian filtering and smoothing are central formalisms across engineering, where real-time inference and detection problems are common [29,45]. We compute formulae from Eq. (3) via the EpiFilter package (see S1 Appendix and [15,28]), which uses optimal forward-backward algorithms, improves on the window-based approach of the last section and maximises the signal-to-noise ratio in R-estimation. We further obtain filtered and smoothed probabilities of resurgence as Embedded Image and Embedded Image. The probability that the epidemic is controlled (i.e., R ≤ 1) is the complement of these expressions. Our main results, which average the above quantities over many simulated Ebola virus and COVID-19 epidemics, are given in Fig 2 and S1 Appendix Fig A, respectively. The simulated incidence curves are provided in S1 Appendix Figs B-C and illustrate the expected differences in case numbers associated with upward and downward R-shifts. We uncover striking differences in the intrinsic ability to infer resurgence versus control in real time.

Fig 2:
Fig 2: Resurgence and control dynamics of Ebola virus.

Using renewal models with the generation time from [46], we simulate 1000 realisations of Ebola virus epidemics (t = 300) with step (A panels) and seasonally (B panels) changing transmissibility (true Rs in black). Top panels show posterior mean estimates from the filtered (Ep[Rs], blue) and smoothed (Eq[Rs], red) distributions from every realisation (computed using EpiFilter [15]). Middle panels average the Kullback-Liebler divergences from those simulations, D(ps|qs), and bottom panels display the overall filtered (Embedded Image, blue), and smoothed (Embedded Image, red) probabilities of resurgence. We find fundamental and striking delays in detecting resurgence, often an order of magnitude longer than those for detecting control or suppression in transmission (see lags between red and blue curves in all relevant panels). Note that the initial rise in Embedded Image of panel A, which precedes the transition in Rs, is due to the influence of the prior distribution (which has a mean above 1) in a period with very few cases. We present the incidence curves that underlie the simulations here in Fig C of the S1 Appendix.

Upward change-points are significantly harder to detect both in terms of accuracy and timing. Discrepancies between ps- and qs-based estimates (the latter benchmark the best realisable performance) are appreciably larger for resurgence than control. While decreases in R can be pinpointed reliably, increases seem fundamentally more difficult to detect. These limits appear to exacerbate with the steepness of the R upswing. We confirm these trends with a detailed simulation study across five infectious diseases in Fig 3. There we alter the steepness, θ, of transmissibility changes and map delays in detecting resurgence and control as a function of the difference in the first time that ps- and qs-based probabilities cross 0.5 (Δt50) and 0.95 (Δt95), normalised by the mean generation time of the disease. We find that lags in detecting resurgence can be at least 5–10 times longer than for detecting control and are of the order of the average intrinsic generation time of the disease.

Fig 3:
Fig 3: Delays in detecting upward and downward changes in R.

We characterise the discrepancies between detecting resurgence and control against the steepness or rate, θ, of changes in transmissibility (Rs), which we model using logistic functions (panel A, steepness increases from blue to red). We compare differences in the probability of detecting resurgence (P(Rs > 1)) or control (P(Rs ≤ 1) under filtered and smoothed estimates (see main text) first crossing thresholds of 0.5 (Δt50) and 0.95 (Δt95) for five infectious diseases (panel B plots their assumed generation time distributions from [2,46,47]). We simulate 1000 epidemics from each disease using renewal models and estimate Rs with EpiFilter [15]. Panels C and D (here colours match panel B, Δt is normalised by the mean generation times of the diseases) show that delays in detecting resurgence (dots with colours indicating the disease) are at least 5– 10 times longer than for detecting control (diamonds with equivalent colours). Our ability to infer even symmetrical transmissibility changes is fundamentally asymmetric, largely due to the differences in case incidence at which those changes usually tend to occur.

Fundamental delays worsen with spatial or demographic heterogeneities

In previous sections we demonstrated that sensitivity to changes in R is asymmetric, and that intrinsic, restrictive limits exist on detecting resurgence in real time, which do not equally inhibit detecting control. While those conclusions apply generally (e.g., across diseases), they do not consider the influence of spatial or demographic heterogeneity. We examine this complexity through a simple but realistic generalisation of the renewal model. Often R-estimates can be computed at small scales (e.g., at the municipality level) via local incidence or more coarsely (e.g., countrywide), using aggregated case counts [3,13]. We can relate these differing scales with the weighted mean in Eq. (4), where the overall (coarse) R at time s,Embedded Image, is a convex sum of finer-scale R contributions from each group (Rs[j] for the jth of p groups) weighted by the epidemic size of that group (as in Eq. (2) we use windows τ(s) for analytic insight). Embedded Image Our choice of groupings is arbitrary and can equally model demographic heterogeneities (e.g., age-specific transmission), where we want to understand how dynamics within the subgroups influence overall spread [7]. Our aim is to ascertain how grouping, which often occurs naturally due to data constraints or a need to succinctly describe the infectious dynamics over a country to aid policymaking or public communication [48], affects resurgence detection. Eq. (4) implies that Embedded Image. Since resurgence will likely first occur within some specific (maybe high risk) group and then propagate to other groups [7], this expression suggests that an initial signal (e.g., if some Rs[j] > 1) could be masked by non-resurging groups (which are, from this perspective, contributing background noise).

As the epidemic size in a resurging group will likely be smaller than those of groups with past epidemics that are now being stabilised or controlled, this exacerbates the sensitivity bounds explored earlier via Eq. (2). We can verify this further loss of sensitivity by examining how the overall posterior distribution depends on those of the component groups as follows, with Embedded Image as a repeated convolution operation and Ωj as the posterior distribution for the jth group. Embedded Image While Eq. (5) holds generally, we assume gamma posterior distributions, leading to statistics analogous to Eq. (2). We plot these sensitivity results at p = 2 and 3 in Fig 4, where group 1 features resurgence and other groups either contain stable or falling incidence. We find that as p grows (and additional distributions convolve to generate Embedded Image) we lose sensitivity (posterior distributions overlap more for a given relative change in incidence Embedded Image.

Fig 4:
Fig 4: Influence of heterogeneities in transmission.

We investigate how differences in transmissibility among groups (e.g., due to demographic or spatial factors) fundamentally limit the ability to detect resurgence from a specific group (in this example group 1 with reproduction number Rs[1]). Panel A shows that the grouped posterior distribution becomes less sensitive to a fixed relative change in group 1 incidence, Embedded Image (the level of change increases from blue to red). Posterior distributions over Embedded Image (the overall reproduction number across groups) are more overlapped (and tighter in variance) as p rises, for fixed Rs[1] (top). Panel B plots how overall resurgence detection probability Embedded Image depends on the weight (α1, top, 0.05–1) and epidemic size (λτ(s)[1], middle, 20–80, p = 2) as well as changes in Rs[3] (bottom, 0.5–1.2, p = 3). Decreases in α1 (red to blue) or λτ(s)[1] mean other groups mask the resurging dynamics in group 1, reducing sensitivity (curves become less steep). In the latter case the Embedded Image (green with solid line at median of λτ(s)[1] range) is always more conservative than P(Rs[1] > 1) (black with solid median line). As Rs[3] falls (red to blue) the ability to detect resurgence also lags relative to that from observing group 1 (black).

Reductions in either the weight (α1), epidemic size (λτ(s)[1]) or other Rs[j ≠ 1], further desensitise the resurgence signals i.e., decrease the gradient of detection probability curves. This is summarised by noting that if Rs[1] = maxj Rs[j], then the sensitivity from Eq. (2) is only matched when the resurging group dominates (α1 ≈ 1) or if other groups have analogous R i.e., Rs[1] ≈ Rs[j]. Delays in detecting resurgence can therefore be severe. Heterogeneity on its own, however, does not force asymmetry between detecting control and resurgence.

Discussion

Probing the performance limits of noisy biological systems has yielded important insights into the real-time estimation and control of parameters in biochemistry and neuroscience [49–51]. Although models from these fields share dynamic similarities with those in epidemiology, there has been relatively little investigation of how real-time estimates of pathogen transmissibility, parametrised by R, might be fundamentally limited. This is surprising since R is among key parameters considered in initiatives aiming to better systematise real-time epidemic response [41,52]. Here we explored what limits may exist on our ability to reliably detect or measure the change-points in R that signify resurgence and control. By using a combination of Bayesian sensitivity analyses and minimum MSE filtering and smoothing algorithms, we discovered striking asymmetries in innate detection sensitivities. We found that, arguably, the most crucial transitions in epidemic transmissibility are possibly the most inherently difficult to detect.

Specifically, resurgence, signified by an increase in R from below to above 1, can possibly be detected only 5–10 times later than an equivalent decrease in R that indicated control (Fig 2, Fig 3 and S1 Appendix Fig A). As this lag can be of the order of the mean generation time of the pathogen under study, even when case reporting is perfect and optimised detection algorithms are applied, this represents a potentially sharp bottleneck to real-time responses for highly contagious diseases. Intuition for this result came from observing that sensitivity to R change-points will weaken (due to noise masking the signal) with declining epidemic sizes or case incidence, and increasing ‘true’ R, both of which likely occur in resurgent settings due to periods of subcritical spread (Eq. (2) and Fig 1). The converse applies to control, which is usually enforced in larger (and less intrinsically noisy) incidence regimes. Furthermore, we found that these latencies and sensitivity issues would only exacerbate when heterogeneous groupings across geography or demography (Eq. (4), (5) and Fig 4) are considered.

An interesting corollary of these results occurs if we consider the detection of an upward shift in R at large incidence. If this increase affects the majority of cases (i.e., Eq. (2) applies), then we would detect it without significant delay because epidemic curves are now inherently less noisy. However, if incidence is large and a resurgence occurs in some subset of the cases (i.e., the upward R-shift is localised to group j and Eq. (5) applies) then we would still face the innate delay of a mean generation time together with further loss of sensitivity due to the cases in groups other than j acting as background noise. This scenario might realistically occur when a new pathogenic variant emerges (e.g., the alpha COVID-19 variant appeared during a high incidence period in the UK [53]) or when specific age groups sustain resurgence (e.g., the 20-49 age group for COVID-19 in the USA [7]). These detection delays limit our ability to rapidly identify and target interventions at resurgent groups. Our work emphasises that correlations among incidence, transmissibility parameters underlying this incidence and heterogeneous groups contributing to that incidence can fundamentally constrain our response sensitivity.

Practical real-time analyses often involve grouping or data aggregation [9,13] and are subject to reporting and other latencies (e.g., if notifications, hospitalisations or deaths are used as proxies for infection incidence), which introduce additive delays on top of those we uncovered [14,54]. Consequently, we argue that while case data may provide robust signals for pinpointing when epidemics are under control (and assessing impacts of interventions), they are insufficient, on their own, to sharply resolve resurgence at low incidence. This does not devalue methods seeking to better characterise real-time R changes [1,2,13,28], but instead contextualises how such inferences should be interpreted when informing policy. Given the intrinsic delays in detecting resurgences, which might associate with critical epidemiological changes such as variants of concern or shifts in population behaviours [6,7], there might be grounds for conservative policies (e.g., those of New Zealand and Australia for COVID-19 [55]) that trade off early interventions against the expense of false alarms. While the value of such policies ultimately depends on many complex economic, political and socio-behavioural factors, our study, together with works that show how lags in enacting interventions can induce drastic costs [33–36], provides a first step towards dissecting some of these trade-offs.

Moreover, our analyses suggest that designing enhanced surveillance systems, which can comprehensively engage and integrate diverse data sources [30,31] may be more important than improving models for processing case data. Fusing multiple and sometimes novel data sources, such as wastewater or cross-sectional viral loads [18,32], may present the only truly realistic means of minimizing the innate bottlenecks to resurgence detection that we have demonstrated. Approaches aimed at improving case-based inference generally correct for reporting biases or propose more robust measures of transmissibility, such as time-varying growth rates [14,41,56]. However, as our study highlights limits that persist at the gold standard of perfect case reporting and, further it is known that under such conditions growth rates and R are equally informative [57], these lines of investigation are unlikely to minimise the detection limits that we have exposed.

There are three main limitations of our results. First, as we only considered renewal model epidemic descriptions with assumed generation times, which predominate real-time R studies, our work necessarily neglects the often-complex contact network structures that can mediate infection spread [58] or lead to intervention-induced generation time changes [39]. However, other analyses using somewhat different approaches to ours (e.g., Hawkes processes [59]) show apparently similar sensitivity asymmetries. There is evidence that renewal models may be as accurate as network models for inferring R [60], while being easier to run and fit in real time. They are also known to be equivalent to various compartmental models [61]. We do not examine the influence of generation time changes, as data on those are rarely available for routine, real-time analyses. However, as the ratio of the resurgence to control lags is 5–10, we expect this asymmetry to be robust to generation time changes, which are relatively smaller [39]. Given the flexibility of our model and that the asymmetry we discovered is contingent on low-incidence data being noisier and typical of resurgence settings, which is a model agnostic point, we expect that the intrinsic limits we have exposed are general and not model artefacts.

Second, while we analysed one common and important definition of resurgence that depends on effective reproduction numbers, other more recent definitions of epidemic re-emergence exist that are linked to complex dynamic characteristics of diseases such as critical slowing down [62]. Our aim was to understand and expose limitations of the most common surveillance data types (incidence) and the most prominent epidemic summary statistics (time-varying or effective reproduction numbers), which are among those informing policy [41], so we did not examine such metrics. Testing to see if these other characteristics also show asymmetry could be an interesting follow-up study but would require different modelling approaches. Last, we did not include any explicit economic modelling. While this is outside the scope of this work it is important to recognise that resurgence detection threshold choices (i.e., how we decide which fluctuations in incidence are actionable) imply some judgment about the relative cost of true positives (timely resurgence detections) versus false alarms [12]. Incorporating explicit cost structures could mean that delays in detecting resurgence are acceptable. We consider this the next investigative step in our aim to probe the limits of real-time performance.

Data Availability

All data and code used to generate the analyses and figures of this work are freely available at: https://github.com/kpzoo/resurgence-detection.

Funding

KVP and CAD acknowledge funding from the MRC Centre for Global Infectious Disease Analysis (reference MR/R015600/1), jointly funded by the UK Medical Research Council (MRC) and the UK Foreign, Commonwealth & Development Office (FCDO), under the MRC/FCDO Concordat agreement and is also part of the EDCTP2 programme supported by the European Union. CAD thanks the UK National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Emerging and Zoonotic Infections in partnership with Public Health England (PHE) for funding (grant HPRU200907). The funders had no role in study design, data collection and analysis, decision to publish, or manuscript preparation.

Supporting Information

S1 Appendix: Methods and Additional Figures. This provides derivations of all equations presented in the main text (key intermediate expressions are given as Eq. (A) and Eq. (B)) and additional Figs A-C. Fig A: Resurgence and control dynamics of COVID-19. We repeat the simulations from Fig 2 but for realisations of COVID-19 epidemics. Fig B: Incidence curves for COVID-19. We present the simulated counts of daily new cases that underlie the results of Fig A. Fig C: Incidence curves for Ebola virus disease. We present the counts of daily new cases that underlie the results of Fig 2 of the main text.

Data availability statement

All data and code used to generate the analyses and figures of this work are freely available at: https://github.com/kpzoo/resurgence-detection.

Author contributions

Conceptualisation, Formal Analysis, Investigation and Methodology, Software and Writing – original draft: KVP. Validation: CAD. Writing – review and editing: KVP and CAD.

Footnotes

  • Reworked figures and clearer and more nuanced discussion of results.

References

  1. 1.↵
    Cauchemez S, Boëlle P-Y, Thomas G, Valleron A-J. Estimating in real time the efficacy of measures to control emerging communicable diseases. Am J Epidemiol. 2006;164: 591– 597. doi:10.1093/aje/kwj274
    OpenUrlCrossRefPubMedWeb of Science
  2. 2.↵
    Cori A, Ferguson NM, Fraser C, Cauchemez S. A new framework and software to estimate time-varying reproduction numbers during epidemics. Am J Epidemiol. 2013;178: 1505– 1512. doi:10.1093/aje/kwt133
    OpenUrlCrossRefPubMed
  3. 3.↵
    The R value and growth rate - GOV.UK [Internet]. [cited 1 Jul 2021]. Available: https://www.gov.uk/guidance/the-r-value-and-growth-rate
  4. 4.↵
    Li Y, Campbell H, Kulkarni D, Harpur A, Nundy M, Wang X, et al. The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries. Lancet Infect Dis. 2021;21: 193–202. doi:10.1016/S1473-3099(20)30785-4
    OpenUrlCrossRefPubMed
  5. 5.↵
    Funk S, Camacho A, Kucharski AJ, Lowe R, Eggo RM, Edmunds WJ. Assessing the performance of real-time epidemic forecasts: A case study of Ebola in the Western Area region of Sierra Leone, 2014-15. PLoS Comput Biol. 2019;15: e1006785. doi:10.1371/journal.pcbi.1006785
    OpenUrlCrossRef
  6. 6.↵
    Sabino EC, Buss LF, Carvalho MPS, Prete CA, Crispim MAE, Fraiji NA, et al. Resurgence of COVID-19 in Manaus, Brazil, despite high seroprevalence. Lancet. 2021;397: 452–455. doi:10.1016/S0140-6736(21)00183-5
    OpenUrlCrossRefPubMed
  7. 7.↵
    Monod M, Blenkinsop A, Xi X, Hebert D, Bershan S, Tietze S, et al. Age groups that sustain resurging COVID-19 epidemics in the United States. Science. 2021;371. doi:10.1126/science.abe8372
    OpenUrlAbstract/FREE Full Text
  8. 8.↵
    Wallinga J, van Boven M, Lipsitch M. Optimizing infectious disease interventions during an emerging epidemic. Proc Natl Acad Sci USA. 2010;107: 923–928. doi:10.1073/pnas.0908491107
    OpenUrlAbstract/FREE Full Text
  9. 9.↵
    Brauner JM, Mindermann S, Sharma M, Johnston D, Salvatier J, Gavenciak T, et al. Inferring the effectiveness of government interventions against COVID-19. Science. 2021;371. doi:10.1126/science.abd9338
    OpenUrlAbstract/FREE Full Text
  10. 10.↵
    Parag KV, Donnelly CA. Adaptive estimation for epidemic renewal and phylogenetic skyline models. Syst Biol. 2020;69: 1163–1179. doi:10.1093/sysbio/syaa035
    OpenUrlCrossRef
  11. 11.
    Dehning J, Zierenberg J, Spitzner FP, Wibral M, Neto JP, Wilczek M, et al. Inferring change points in the spread of COVID-19 reveals the effectiveness of interventions. Science. 2020;369. doi:10.1126/science.abb9789
    OpenUrlAbstract/FREE Full Text
  12. 12.↵
    Johnson P, Moriarty J, Peskir G. Detecting changes in real-time data: a user’s guide to optimal detection. Philos Trans A, Math Phys Eng Sci. 2017;375. doi:10.1098/rsta.2016.0298
    OpenUrlCrossRefPubMed
  13. 13.↵
    Abbott S, Hellewell J, Thompson RN, Sherratt K, Gibbs HP, Bosse NI, et al. Estimating the time-varying reproduction number of SARS-CoV-2 using national and subnational case counts. Wellcome Open Res. 2020;5: 112. doi:10.12688/wellcomeopenres.16006.2
    OpenUrlCrossRef
  14. 14.↵
    Gostic KM, McGough L, Baskerville EB, Abbott S, Joshi K, Tedijanto C, et al. Practical considerations for measuring the effective reproductive number, Rt. PLoS Comput Biol. 2020;16: e1008409. doi:10.1371/journal.pcbi.1008409
    OpenUrlCrossRefPubMed
  15. 15.↵
    Parag KV. Improved estimation of time-varying reproduction numbers at low case incidence and between epidemic waves. PLoS Comput Biol. 2021;17: e1009347.doi.org/10.1371/journal.pcbi.1009347
    OpenUrlCrossRef
  16. 16.↵
    Riley S, Ainslie KEC, Eales O, Walters CE, Wang H, Atchison C, et al. Resurgence of SARS-CoV-2: Detection by community viral surveillance. Science. 2021;372: 990–995. doi:10.1126/science.abf0874
    OpenUrlAbstract/FREE Full Text
  17. 17.
    du Plessis L, McCrone JT, Zarebski AE, Hill V, Ruis C, Gutierrez B, et al. Establishment and lineage dynamics of the SARS-CoV-2 epidemic in the UK. Science. 2021;371: 708– 712. doi:10.1126/science.abf2946
    OpenUrlAbstract/FREE Full Text
  18. 18.↵
    Hay JA, Kennedy-Shaffer L, Kanjilal S, Lennon NJ, Gabriel SB, Lipsitch M, et al. Estimating epidemiologic dynamics from cross-sectional viral load distributions. Science. 2021; doi:10.1126/science.abh0635
    OpenUrlAbstract/FREE Full Text
  19. 19.↵
    Casella F. Can the COVID-19 Epidemic Be Controlled on the Basis of Daily Test Reports? IEEE Control Syst Lett. 2021;5: 1079–1084. doi:10.1109/LCSYS.2020.3009912
    OpenUrlCrossRef
  20. 20.
    Holmdahl I, Buckee C. Wrong but Useful -What Covid-19 Epidemiologic Models Can and Cannot Tell Us. N Engl J Med. 2020;383: 303–305. doi:10.1056/NEJMp2016822
    OpenUrlCrossRefPubMed
  21. 21.↵
    Fraser C, Riley S, Anderson RM, Ferguson NM. Factors that make an infectious disease outbreak controllable. Proc Natl Acad Sci USA. 2004;101: 6146–6151. doi:10.1073/pnas.0307506101
    OpenUrlAbstract/FREE Full Text
  22. 22.↵
    Drake JM. Limits to forecasting precision for outbreaks of directly transmitted diseases. PLoS Med. 2006;3: e3. doi:10.1371/journal.pmed.0030003
    OpenUrlCrossRefPubMed
  23. 23.
    Kahn R, Peak CM, Fernández-Gracia J, Hill A, Jambai A, Ganda L, et al. Incubation periods impact the spatial predictability of cholera and Ebola outbreaks in Sierra Leone. Proc Natl Acad Sci USA. 2020;117: 5067–5073. doi:10.1073/pnas.1913052117
    OpenUrlAbstract/FREE Full Text
  24. 24.
    Parag KV. Sub-spreading events limit the reliable elimination of heterogeneous epidemics. J R Soc Interface. 2021;18: 20210444. doi:10.1098/rsif.2021.0444
    OpenUrlCrossRef
  25. 25.↵
    Scarpino S, Petri G. On the predictability of infectious disease outbreaks. Nat Comms. 2019;10.
  26. 26.↵
    Mercer GN, Glass K, Becker NG. Effective reproduction numbers are commonly overestimated early in a disease outbreak. Stat Med. 2011;30: 984–994. doi:10.1002/sim.4174
    OpenUrlCrossRefPubMed
  27. 27.
    Britton T, Scalia Tomba G. Estimation in emerging epidemics: biases and remedies. J R Soc Interface. 2019;16: 20180670. doi:10.1098/rsif.2018.0670
    OpenUrlCrossRef
  28. 28.↵
    Parag KV, Cowling BJ, Donnelly CA. Deciphering early-warning signals of SARS-CoV-2 elimination and resurgence from limited data at multiple scales. J R Soc Interface. 2021; 18: 20210569. doi.org/10.1098/rsif.2021.0569
    OpenUrlCrossRef
  29. 29.↵
    Sarrka S. Bayesian Filtering and Smoothing. Cambridge, UK: Cambridge University Press; 2013.
  30. 30.↵
    Overview of Syndromic Surveillance What is Syndromic Surveillance? [Internet]. [cited 19 Aug 2021]. Available: https://www.cdc.gov/mmwr/preview/mmwrhtml/su5301a3.htm
  31. 31.↵
    Syndromic surveillance: systems and analyses - GOV.UK [Internet]. [cited 19 Aug 2021]. Available: https://www.gov.uk/government/collections/syndromic-surveillance-systems-and-analyses
  32. 32.↵
    Daughton CG. Wastewater surveillance for population-wide Covid-19: The present and future. Sci Total Environ. 2020;736: 139631. doi:10.1016/j.scitotenv.2020.139631
    OpenUrlCrossRef
  33. 33.↵
    Lipsitch M, Finelli L, Heffernan RT, Leung GM, Redd SC, 2009 H1n1 Surveillance Group. Improving the evidence base for decision making during a pandemic: the example of 2009 influenza A/H1N1. Biosecur Bioterror. 2011;9: 89–115. doi:10.1089/bsp.2011.0007
    OpenUrlCrossRefPubMed
  34. 34.
    Wallinga J, Teunis P. Different epidemic curves for severe acute respiratory syndrome reveal similar impacts of control measures. Am J Epidemiol. 2004;160: 509–516. doi:10.1093/aje/kwh255
    OpenUrlCrossRefPubMedWeb of Science
  35. 35.
    Pei S, Kandula S, Shaman J. Differential effects of intervention timing on COVID-19 spread in the United States. Sci Adv. 2020;6. doi:10.1126/sciadv.abd6370
    OpenUrlFREE Full Text
  36. 36.↵
    Morris DH, Rossine FW, Plotkin JB, Levin SA. Optimal, near-optimal, and robust epidemic control. Commun Phys. 2021;4: 78. doi:10.1038/s42005-021-00570-y
    OpenUrlCrossRef
  37. 37.↵
    Fraser C. Estimating individual and household reproduction numbers in an emerging epidemic. PLoS One. 2007;2: e758. doi:10.1371/journal.pone.0000758
    OpenUrlCrossRefPubMed
  38. 38.↵
    Wallinga J, Lipsitch M. How generation intervals shape the relationship between growth rates and reproductive numbers. Proc R Soc B. 2007;274: 599–604.
    OpenUrlCrossRefPubMedWeb of Science
  39. 39.↵
    Ali ST, Wang L, Lau EHY, Xu X-K, Du Z, Wu Y, et al. Serial interval of SARS-CoV-2 was shortened over time by nonpharmaceutical interventions. Science. 2020;369: 1106–1109. doi:10.1126/science.abc9004
    OpenUrlAbstract/FREE Full Text
  40. 40.↵
    Parag KV, Donnelly CA. Using information theory to optimise epidemic models for real-time prediction and estimation. PLoS Comput Biol. 2020;16: e1007990. doi:10.1371/journal.pcbi.1007990
    OpenUrlCrossRef
  41. 41.↵
    summary E. Reproduction number (R) and growth rate (r) of the COVID-19 epidemic in the UK: methods of.
  42. 42.↵
    Grunwald P. The Minimum Description Length Principle. The MIT Press; 2007.
  43. 43.↵
    Lehmann EL, Casella G. Theory of point estimation. New York: Springer-Verlag; 1998. doi:10.1007/b98854
    OpenUrlCrossRef
  44. 44.↵
    Guo D, Shamai S, Verdu S. Mutual information and conditional mean estimation in poisson channels. IEEE Trans Inform Theory. 2008;54: 1837–1849. doi:10.1109/TIT.2008.920206
    OpenUrlCrossRef
  45. 45.↵
    Snyder D, Miller M. Random Point Processes in Time and Space. 2nd ed. Springer-Verlag; 1991.
  46. 46.↵
    Van Kerkhove MD, Bento AI, Mills HL, Ferguson NM, Donnelly CA. A review of epidemiological parameters from Ebola outbreaks to inform early public health decision-making. Sci Data. 2015;2: 150019. doi:10.1038/sdata.2015.19
    OpenUrlCrossRefPubMed
  47. 47.↵
    Ferguson N, Laydon D, Nedjati-Gilani G, Others. Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand. Imperial College London; 2020.
  48. 48.↵
    Cevik M, Marcus JL, Buckee C, Smith TC. SARS-CoV-2 transmission dynamics should inform policy. Clin Infect Dis. 2020; doi:10.1093/cid/ciaa1442
    OpenUrlCrossRefPubMed
  49. 49.↵
    Bialek W, Setayeshgar S. Physical limits to biochemical signaling. Proc Natl Acad Sci USA. 2005;102: 10040–10045. doi:10.1073/pnas.0504321102
    OpenUrlAbstract/FREE Full Text
  50. 50.
    Parag KV, Vinnicombe G. Point process analysis of noise in early invertebrate vision. PLoS Comput Biol. 2017;13: e1005687. doi:10.1371/journal.pcbi.1005687
    OpenUrlCrossRef
  51. 51.↵
    Lestas I, Vinnicombe G, Paulsson J. Fundamental Limits on the Supression of Molecular Fluctuations. Nature. 2010;467: 174–8.
    OpenUrlCrossRefPubMedWeb of Science
  52. 52.↵
    Polonsky J, Baidjoe A, Kamvar Z, Others. Outbreak analytics: a developing data science for informing the response to emerging pathogens. Phil Trans R Soc B. 2019;374: 20180276.
    OpenUrlCrossRefPubMed
  53. 53.↵
    Volz E, Mishra S, Chand M, Barrett JC, The COVID-19 Genomics UK (COG-UK) consortium, Johnson R, et al. Assessing transmissibility of SARS-CoV-2 lineage B.1.1.7 in England. Nature. 2021; doi:10.1038/s41586-021-03470-x
    OpenUrlCrossRefPubMed
  54. 54.↵
    Goldstein E, Dushoff J, Ma J, Plotkin JB, Earn DJD, Lipsitch M. Reconstructing influenza incidence by deconvolution of daily mortality time series. Proc Natl Acad Sci USA. 2009;106: 21825–21829. doi:10.1073/pnas.0902958106
    OpenUrlAbstract/FREE Full Text
  55. 55.↵
    Baker MG, Wilson N, Blakely T. Elimination could be the optimal response strategy for covid-19 and other emerging pandemic diseases. BMJ. 2020;371: m4907. doi:10.1136/bmj.m4907
    OpenUrlFREE Full Text
  56. 56.↵
    Pellis L, Scarabel F, Stage HB, Overton CE, Chappell LHK, Fearon E, et al. Challenges in control of COVID-19: short doubling time and long delay to effect of interventions. Philos Trans R Soc Lond B, Biol Sci. 2021;376: 20200264. doi:10.1098/rstb.2020.0264
    OpenUrlCrossRef
  57. 57.↵
    Parag KV, Thompson RN, Donnelly CA. Are epidemic growth rates more informative than reproduction numbers? medRxiv. 2021; doi:10.1101/2021.04.15.21255565
    OpenUrlAbstract/FREE Full Text
  58. 58.↵
    Watts DJ, Muhamad R, Medina DC, Dodds PS. Multiscale, resurgent epidemics in a hierarchical metapopulation model. Proc Natl Acad Sci USA. 2005;102: 11157–11162. doi:10.1073/pnas.0501226102
    OpenUrlAbstract/FREE Full Text
  59. 59.↵
    Koyama S, Horie T, Shinomoto S. Estimating the time-varying reproduction number of COVID-19 with a state-space method. PLoS Comput Biol. 2021;17: e1008679. doi:10.1371/journal.pcbi.1008679
    OpenUrlCrossRef
  60. 60.↵
    Liu Q-H, Ajelli M, Aleta A, Merler S, Moreno Y, Vespignani A. Measurability of the epidemic reproduction number in data-driven contact networks. Proc Natl Acad Sci USA. 2018;115: 12680–12685. doi:10.1073/pnas.1811115115
    OpenUrlAbstract/FREE Full Text
  61. 61.↵
    Champredon D, Dushoff J, Earn DJD. Equivalence of the Erlang-Distributed SEIR Epidemic Model and the Renewal Equation. SIAM J Appl Math. 2018;78: 3258–3278. doi:10.1137/18M1186411
    OpenUrlCrossRef
  62. 62.↵
    Drake JM, Brett TS, Chen S, Epureanu BI, Ferrari MJ, Marty É, et al. The statistics of epidemic transitions. PLoS Comput Biol. 2019;15: e1006917. doi:10.1371/journal.pcbi.1006917
    OpenUrlCrossRef
Back to top
Posted February 11, 2022.
Download PDF

Supplementary Material

Data/Code
Email
Fundamental limits on inferring epidemic resurgence in real time using effective reproduction numbers
Kris V. Parag, Christl A. Donnelly
medRxiv 2021.09.08.21263270; doi: https://doi.org/10.1101/2021.09.08.21263270
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools

Subject Area