The Effect of Large-Scale Anti-Contagion Policies on the COVID-19 Pandemic

Solomon Hsiang; Daniel Allen; Sébastien Annan-Phan; Kendon Bell; Ian Bolliger; Trinetta Chong; Hannah Druckenmiller; Luna Yue Huang; Andrew Hultgren; Emma Krasovich; Peiley Lau; Jaecheol Lee; Esther Rolf; Jeanette Tseng; Tiffany Wu

doi:10.1101/2020.03.22.20040642

Abstract

Governments around the world are responding to the novel coronavirus (COVID-19) pandemic¹ with unprecedented policies designed to slow the growth rate of infections. Many actions, such as closing schools and restricting populations to their homes, impose large and visible costs on society, but their benefits cannot be directly observed and are currently understood only through process-based simulations.^2–4 Here, we compile new data on 1,717 local, regional, and national non-pharmaceutical interventions deployed in the ongoing pandemic across localities in China, South Korea, Italy, Iran, France, and the United States (US). We then apply reduced-form econometric methods, commonly used to measure the effect of policies on economic growth, ^5,6 to empirically evaluate the effect that these anti-contagion policies have had on the growth rate of infections. In the absence of policy actions, we estimate that early infections of COVID-19 exhibit exponential growth rates of roughly 38% per day. We find that anti-contagion policies have significantly and substantially slowed this growth. Some policies have different impacts on different populations, but we obtain consistent evidence that the policy packages now deployed are achieving large, beneficial, and measurable health outcomes. We estimate that across these six countries, interventions prevented or delayed on the order of 62 million confirmed cases, corresponding to averting roughly 530 million total infections. These findings may help inform whether or when these policies should be deployed, intensified, or lifted, and they can support decision-making in the other 180+ countries where COVID-19 has been reported.⁷

Introduction

The COVID-19 pandemic is forcing societies worldwide to make consequential policy decisions with limited information. After containment of the initial outbreak failed, attention turned to implementing non-pharmaceutical interventions designed to slow contagion of the virus. In general, these policies aim to decrease virus transmission by reducing contact among individuals within or between populations, such as by closing restaurants or restricting travel, thereby slowing the spread of COVID-19 to a manageable rate. These large-scale anti-contagion policies are informed by epidemiological simulations^{2, 4, 8, 9} and a small number of natural experiments in past epidemics.¹⁰ However, the actual effects of these policies on infection rates in the ongoing pandemic are unknown. Because the modern world has never confronted this pathogen, nor deployed anti-contagion policies of such scale and scope, it is crucial that direct measurements of policy impacts be used alongside numerical simulations in current decision-making.

Societies around the world are weighing whether the health benefits of anti-contagion policies are worth their social and economic costs. Many of these costs are plainly seen; for example, business restrictions increase unemployment and school closures impact educational outcomes. It is therefore not surprising that some populations have hesitated before implementing such dramatic policies, especially when their costs are visible while their health benefits – infections and deaths that would have occurred but instead were avoided or delayed – are unseen. Our objective is to measure the direct health benefits of these policies; specifically, how much these policies slowed the growth rate of infections. To do this, we compare the growth rate of infections within hundreds of sub-national regions before and after each of these policies is implemented locally. Intuitively, each administrative unit observed just prior to a policy deployment serves as the “control” for the same unit in the days after it receives a policy “treatment” (see Supplementary Information for accounts of these deployments). Our hope is to learn from the recent experience of six countries where early spread of the virus triggered large-scale policy actions, in part so that societies and decision-makers in the remaining 180+ countries can access this information.

Here we directly estimate the effects of 1, 717 local, regional, and national policies on the growth rate of infections across localities within China, France, Iran, Italy, South Korea, and the US (see Figure 1 and Supplementary Table 1). We compile subnational data on daily infection rates, changes in case definitions, and the timing of policy deployments, including (1) travel restrictions, (2) social distancing through cancellations of events and suspensions of educational/commercial/religious activities, (3) quarantines and lockdowns, and (4) additional policies such as emergency declarations and expansions of paid sick leave, from the earliest available dates to April 6, 2020 (see Supplementary Notes, also Extended Data Fig. 1). During this period, populations remained almost entirely susceptible to COVID-19, causing the natural spread of infections to exhibit almost perfect exponential growth.^{11, 12} The rate of this exponential growth could change daily, determined by epidemiological factors, such as disease infectivity, as well as policies that alter behavior.^{9, 11, 13} Because policies were deployed while the epidemic unfolded, we can estimate their effects empirically. To do this, we examine how the daily growth rate of infections in each locality changes in response to the collection of ongoing policies applied to that locality on that day.

Figure 1: Data on COVID-19 infections and large-scale anti-contagion policies.

Left: Daily cumulative confirmed cases of COVID-19 (solid black line, left axis) and deaths (dashed black line) over time. Vertical lines are deployments of anti-contagion policies, with height indicating the number of administrative units instituting a policy that day (right axis). For display purposes only, ≤ 5 policy types are shown per country and missing case data are imputed unless all sub-national units are missing. Right: Maps of cumulative confirmed cases by administrative unit on the last date of each sample.

Methods Summary

We employ well-established “reduced-form” econometric techniques^{5, 14} commonly used to measure the effects of events^{6, 15} on economic growth rates. Similar to early COVID-19 infections, economic output generally increases exponentially with a variable rate that can be affected by policies and other conditions. Here, this technique aims to measure the total magnitude of the effect of changes in policy, without requiring explicit prior information about fundamental epidemiological parameters or mechanisms, many of which remain uncertain in the current pandemic. Rather, the collective influence of these factors is empirically recovered from the data without modeling their individual effects explicitly (see Methods). Prior work on influenza, ¹⁶ for example, has shown that such statistical approaches can provide important complementary information to process-based models.

To construct the dependent variable, we transform location-specific, subnational time-series data on infections into first-differences of their natural logarithm, which is the per-day growth rate of infections (see Methods). We use data from first- or second-level administrative units and data on active or cumulative cases, depending on availability (see Supplementary Information). We employ widely-used panel regression models^{5, 14} to estimate how the daily growth rate of infections changes over time within a location when different combinations of large-scale policies are enacted (see Methods). Our econometric approach accounts for differences in the baseline growth rate of infections across subnational locations, which may be affected by time-invariant characteristics, such as demographics, socio-economic status, culture, and health systems; it accounts for systematic patterns in growth rates within countries unrelated to policy, such as the effect of the work-week; it is robust to systematic under-surveillance specific to each subnational unit; and it accounts for changes in procedures to diagnose positive cases (see Methods and Supplementary Information).

Results

We estimate that in the absence of policy, early infection rates of COVID-19 grow 43% per day on average across these six countries (Standard Error [SE]= 5%), implying a doubling time of approximately 2 days. Country-specific estimates range from 34% per day in the US (SE= 7%) to 68% per day in Iran (SE= 9%). We cannot determine if the high estimate for Iran results from true epidemiological differences, data quality issues (see Methods), the concurrence of the initial outbreak with a major religious holiday and pilgrimage (see Supplementary Notes), or sampling variability. Excluding Iran, the average growth rate is 38% per day (SE= 5%). Growth rates in all five other countries are independently estimated to be very near this value (Figure 2a). These estimated values differ from observed average growth rates because the latter are confounded by the effects of policy. These growth rates are not driven by the expansion of testing or increasing rates of case detection (see Methods and Extended Data Fig. 2) nor by data from individual regions (Extended Data Fig. 3).

Figure 2: Empirical estimates of unmitigated COVID-19 infection growth rates and the effect of anti-contagion policies. Markers are country-specific estimates, whiskers are 95% CI. Columns report effect sizes as a change in the continuous-time growth rate (95% CI in parentheses) and the day-over-day percentage growth rate. (a) Estimates of daily COVID-19 infection growth rates in the absence of policy (dashed lines = averages with and without Iran, both excluding Wuhan-specific estimate). (b) Estimated combined effect of all policies on infection growth rates. (c) Estimated effects of individual policies or policy groups on the daily growth rate of infections, jointly estimated and ordered roughly chronologically within each country. *Reported effect of home isolation” includes effects of other implied policies (see Methods). China: N=3669; South Korea: N=595, Italy: N=2898, Iran: N=548, France: N=270, US: N=1238.

Some prior analyses of pre-intervention infections in Wuhan suggest slower growth rates (doubling every 5-7 days)^{17, 18} using data collected before national standards for diagnosis and case definitions were first issued by the Chinese government on January 15, 2020.¹⁹ However, case data in Wuhan from before this date contain multiple irregularities: the cumulative case count decreased on January 9; no new cases were reported during January 9-15; and there were concerns that information about the outbreak was suppressed²⁰ (see Supplementary Table 2). When we remove these problematic data, utilizing a shorter but more reliable pre-intervention time series from Wuhan (January 16–21), we recover a growth rate of 43% per day (SE= 3%, doubling every 2 days) consistent with results from all other countries except Iran (Figure 2a, Supplementary Table 3).

During the early stages of an epidemic, a large proportion of the population remains susceptible to the virus, and if the spread of the virus is left uninhibited by policy or behavioral change, exponential growth continues until the fraction of the susceptible population declines meaningfully.^{11, 13, 21, 22} After correcting for estimated rates of case-detection, ²³ we compute that the minimum susceptible fraction across administrative units in our sample is 72% of the total population (Cremona, Italy) and 87% of units would likely be in a regime of uninhibited exponential growth (> 95% susceptible) if policies were removed on the last date of our sample.

Consistent with predictions from epidemiological models, ^{2, 10, 24} we find that the combined effect of policies within each country reduces the growth rate of infections by a substantial and statistically significant amount (Figure 2b, Supplementary Table 3). For example, a locality in France with a baseline growth rate of 0.33 (national average) that fully deployed all policy actions used in France would be expected to lower its daily growth rate by -0.17 to a growth rate of 0.16. In general, the estimated total effects of policy packages are large enough that they can in principle offset a large fraction of, or even eliminate, the baseline growth rate of infections—although in several countries, many localities have not deployed the full set of policies. Overall, the estimated effects of all policies combined are generally insensitive to withholding regional (i.e. state- or province-level) blocks of data from the sample (Extended Data Fig. 3).

In China, only three policies were enacted across 116 cities early in a seven week period, providing us with sufficient data to empirically estimate how the effects of these policies evolved over time without making assumptions about the timing of these effects (see Methods and Fig. 2b). We estimate that the combined effect of these policies reduced the growth rate of infections by −0.026 (SE= 0.046) in the first week following their deployment, increasing substantially in the second week to -0.20 (SE= 0.049), and essentially stabilizing in the third week near −0.28 (SE= 0.047). In other countries, we lack sufficient data to estimate these temporal dynamics explicitly and only report the average pooled effect of policies across all days following their deployment (see Methods). If other countries have transient responses similar to China, we would expect effects in the first week following deployment to be smaller in magnitude than the average effect we report. In Extended Data Fig. 5a and Supplementary Methods Section 3, we explore how our estimates would change if we impose the assumption that policies cannot affect infection growth rates until after a fixed number of days; however, we do not find evidence this improves model fit.

The estimates above (Figure 2b) capture the superposition of all policies deployed in each country, i.e., they represent the average effect of policies that we would expect to observe if all policies enacted anywhere in each country were implemented simultaneously in a single region of that country. We also estimate the effects of individual policies or clusters of policies (Figure 2c) that are grouped based on either their similarity in goal (e.g., library and museum closures) or timing (e.g., policies deployed simultaneously). Our estimates for these individual effects tend to be statistically noisier than the estimates for all policies combined. Some estimates for the same policy differ between countries, perhaps because policies are not implemented identically or because populations behave differently. Nonetheless, 22 out of 29 point estimates indicate that individual policies are likely contributing to reducing the growth rate of infections. Seven policies (one in South Korea, two in Italy, and four in the US) have point estimates that are positive, six of which are small in magnitude (< 0.1) and not statistically different from zero (5% level). Consistent with greater overall uncertainty in these dis-aggregated estimates, some in China, South Korea, Italy, and France are somewhat more sensitive to withholding regional blocks of data (Extended Data Fig. 4), but remain broadly robust to assuming a constant delayed effect of all policies (Extended Data Fig. 5b).

Based on these results, we find that the deployment of anti-contagion policies in all six countries significantly and substantially slowed the pandemic. We combine the estimates above with our data on the timing of the 1, 717 policy deployments to estimate the total effect of all policies across the dates in our sample. To do this, we use our estimates to predict the growth rate of infections in each locality on each day, given the actual policies in effect at that location on that date (Figure 3, blue markers). We then use the same model to predict what counterfactual growth rates would be on that date if the effects of all policies were removed (Figure 3, red markers), which we call the “no-policy scenario.” The difference between these two predictions is our estimated effect that all deployed policies had on the growth rate of infections. During our sample, we estimate that all policies combined slowed the average growth rate of infections by −0.252 per day (SE= 0.045, p< 0.001) in China, −0.248 (SE= 0.089, p< 0.01) in South Korea, −0.24 (SE= 0.068, p< 0.001) in Italy, −0.355 (SE= 0.063, p< 0.001) in Iran, −0.123 (SE= 0.019, p< 0.001) in France and -0.084 (SE= 0.03, p< 0.01) in the US. These results are robust to modeling the effects of policies without grouping them (Extended Data Fig. 6a and Supplementary Table 4) or assuming a delayed effect of policy on infection growth rates (Supplementary Table 5).

Figure 3: Estimated infection growth rates based on actual anti-contagion policies and in a “no policy” counterfactual scenario.

Predicted daily growth rates of active (China, South Korea) or cumulative (all others) COVID-19 infections based on the observed timing of all policy deployments within each country (blue) and in a scenario where no policies were deployed (red). The difference between these two predictions is our estimated effect of actual anti-contagion policies on the growth rate of infections. Small markers are daily estimates for sub-national administrative units (vertical lines are 95% CI). Large markers are national averages. Black circles are observed daily changes in log(infections), averaged across administrative units. Sample sizes are the same as Figure 2.

The number of COVID-19 infections on a date depends on the growth rate of infections on all prior days. Thus, persistent reductions in growth rates have a compounding effect on infections, until growth is slowed by a shrinking susceptible population. To provide a sense of scale for our results, we integrate the growth rate of infections in each locality from Figure 3 to estimate cumulative infections, both with actual anti-contagion policies and in the no-policy counterfactual scenario. To account for the declining susceptible population in each administrative unit, we couple our econometric estimates of the effects of policies with a Susceptible-Infected-Removed (SIR) model^{11, 13} that adjusts the susceptible population in each administrative unit based on estimated case-detection rates^{23, 25} (see Methods). This allows us to extend our projections beyond the initial exponential growth phase of infections, a threshold that many localities cross in our no-policy scenario.

Our results suggest that ongoing anti-contagion policies have already substantially reduced the number of COVID-19 infections observed in the world today (Figure 4). Our central estimates suggest that there would be roughly 37 million more cumulative confirmed cases (corresponding to 285 million more total infections, including the confirmed cases) in China, 11.5 million more confirmed cases in South Korea (38 million total infections), 2.1 million more confirmed cases in Italy (49 million total infections), 5 million more confirmed cases in Iran (54 million total infections), 1.4 million more confirmed cases in France (45 million total infections), and 4.8 million more confirmed cases (60 million total infections) in the US had these countries never enacted any anti-contagion policies since the start of the pandemic. The magnitudes of these impacts partially reflect the timing, intensity, and extent of policy deployment (e.g., how many localities deployed policies), and the duration for which they have been applied. Several of these estimates are subject to large statistical uncertainties (see intervals in Figure 4). Sensitivity tests (Extended Data Fig. 7) that assume a range of plausible alternative parameter values relating to disease dynamics, such as incorporating a Susceptible-Exposed-Infected-Removed (SEIR) model, suggest that interventions may have reduced the severity of the outbreak by a total of 55-66 million confirmed cases over the dates in our sample (central estimates). Sensitivity tests varying the assumed infection-fatality ratio (Supplementary Table 6) suggest a corresponding range of 46-77 million confirmed cases (490-580 million total infections).

Figure 4: Estimated cumulative confirmed COVID-19 infections with and without anti-contagion policies.

The predicted cumulative number of confirmed COVID-19 infections based on actual policy deployments (blue) and in the no-policy counterfactual scenario (red). Shaded areas show uncertainty based on 1, 000 simulations where empirically estimated parameters are resampled from their joint distribution (dark = inner 70% of predictions; light = inner 95%). Black dotted line is observed cumulative infections. Infections are not projected for administrative units that never report infections in the sample, but which might have experienced infections in a no-policy scenario.

Discussion

Our empirical results indicate that large-scale anti-contagion policies are slowing the COVID-19 pandemic. Because infection rates in the countries we study would have initially followed rapid exponential growth had no policies been applied, our results suggest that these policies have provided large health benefits. For example, we estimate that there would be roughly 465× the observed number of confirmed cases in China, 17× in Italy, and 14× in the US by the end of our sample if large-scale anti-contagion policies had not been deployed. Consistent with process-based simulations of COVID-19 infections, ^{2, 4, 8, 9, 22, 26} our analysis of existing policies indicates that seemingly small delays in policy deployment likely produced dramatically different health outcomes.

While the limitations of available data pose challenges to our analysis, our aim is to use what data exist to estimate the first-order impacts of unprecedented policy actions in an ongoing global crisis. As more data become available, related findings will become more precise and may capture more complex interactions. Furthermore, this analysis does not account for interactions between populations in nearby localities, ¹³ nor mobility networks.^{3, 4, 8, 9} Nonetheless, we hope these results can support critical decision-making, both in the countries we study and in the other 180+ countries where COVID-19 infections have been reported.⁷

A key advantage of our reduced-form “top down” statistical approach is that it captures the real-world behavior of affected populations without requiring that we explicitly model underlying mechanisms and processes. This is useful in the current pandemic where many process-related parameters remain uncertain. However, our results cannot and should not be interpreted as a substitute for “bottom up” process-based epidemiological models specifically designed to provide guidance in public health crises. Rather, our results complement existing models, for example, by helping to calibrate key model parameters. We believe both forward-looking simulations and backward-looking empirical evaluations should be used to inform decision-making.

Our analysis measures changes in local infection growth rates associated with changes in anticontagion policies. A necessary condition for this association to be interpreted as the plausibly causal effect of these policies is that the timing of policy deployment is independent of infection growth rates.¹⁴ This assumption is supported by established epidemiological theory^{11, 13, 27} and evidence, ^{28, 29} which indicate that infections in the absence of policy will grow exponentially early in the epidemic, implying that pre-policy infection growth rates should be constant over time and therefore uncorrelated with the timing of policy deployment. Further, scientific guidance to decision-makers early in the current epidemic explicitly projected constant growth rates in the absence of anti-contagion measures, limiting the possibility that anticipated changes in natural growth rates affected decision-making.^{2, 22, 30, 31} In practice, policies tended to be deployed in response to high total numbers of cases (e.g. in France), ³² in response to outbreaks in other regions (e.g. in China, South Korea, and Iran), ³³ after delays due to political constraints (e.g. in the US and Italy), and often with timing that coincided with arbitrary events, like weekends or holidays (see Supplementary Notes for detailed chronologies).

Our analysis accounts for documented changes in COVID-19 testing procedures and availability, as well as differences in case-detection across locations; however, unobserved trends in case-detection could affect our results (see Methods). We analyze estimated case-detection trends²³ (Extended Data Fig. 2), finding that this potential bias is small, possibly elevating our estimated no-policy growth rates by 0.022 (7%) on average.

It is also possible that changing public knowledge during the period of our study affects our results. If individuals alter behavior in response to new information unrelated to anti-contagion policies, such as seeking out online resources, this could alter the growth rate of infections and thus affect our estimates. If increasing availability of information reduces infection growth rates, it would cause us to overstate the effectiveness of anti-contagion policies. We note, however, that if public knowledge is increasing in response to policy actions, such as through news reports, then it should be considered a pathway through which policies alter infection growth, not a form of bias. Investigating these potential effects is beyond the scope of this analysis, but it is an important topic for future investigations.

Finally, our analysis focuses on confirmed infections, but other outcomes, such as hospitalizations or deaths, are also of policy interest. Future work on these outcomes may require additional modeling approaches because they are relatively more context-and state-dependent. Nonetheless, we experimentally implement our approach on the daily growth rate of hospitalizations in France, where hospitalization data is available at the granularity of this study. We find that the total estimated effect of anti-contagion policies on the growth rate of hospitalizations is similar to our estimates for infection growth rates (Extended Data Fig. 6c).

Data Availability

All data and code used in this analysis are available at https://github.com/bolliger32/gpl-covid. Updates are posted at http://www.globalpolicy.science/covid19.

Methods

Data Collection and Processing

We provide a brief summary of our data collection processes here (see the Supplementary Notes for more details, including access dates). Epidemiological, case definition/testing regime, and policy data for each of the six countries in our sample were collected from a variety of in-country data sources, including government public health websites, regional newspaper articles, and crowd-sourced information on Wikipedia. The availability of epidemiological and policy data varied across the six countries, and preference was given to collecting data at the most granular administrative unit level. The country-specific panel datasets are at the region level in France, the state level in the US, the province level in South Korea, Italy and Iran, and the city level in China. Due to data availability, the sample dates differ across countries: in China we use data from January 16 - March 5, 2020; in South Korea from February 17 - April 6, 2020; in Italy from February 26 - April 6, 2020; in Iran from February 27 - March 22, 2020; in France from February 29 - March 25, 2020; and in the US from March 3 - April 6, 2020. Below, we describe our data sources.

China

We acquired epidemiological data from an open source GitHub project³⁴ that scrapes time series data from Ding Xiang Yuan. We extended this dataset back in time to January 10, 2020 by manually collecting official daily statistics from the central and provincial (Hubei, Guangdong, and Zhejiang) Chinese government websites. We compiled policies by collecting data on the start dates of travel bans and lockdowns at the city-level from the “2020 Hubei lockdowns” Wikipedia page³⁵ and various other news reports. We suspect that most Chinese cities have implemented at least one anti-contagion policy due to their reported trends in infections; as such, we dropped cities where we could not identify a policy deployment date to avoid miscategorizing the policy status of these cities. Thus our results are only representative for the sample of 116 cities for which we obtained policy data.

South Korea

We manually collected and compiled the epidemiological dataset in South Korea, based on provincial government reports, policy briefings, and news articles. We compiled policy actions from news articles and press releases from the Korean Centers for Disease Control and Prevention (KCDC), the Ministry of Foreign Affairs, and local governments’ websites.

Iran

We used epidemiological data from the table “New COVID-19 cases in Iran by province”³⁶ in the “2020 coronavirus pandemic in Iran” Wikipedia article, which were compiled from data provided on the Iranian Ministry of Health website (in Persian). We relied on news media reporting and two timelines of pandemic events in Iran^37,36 to collate policy data. From March 2-3, Iran did not report subnational cases. Around this period the country implemented three national policies: a recommendation against local travel (3/1), work from home for government employees (3/3), and school closure (3/5). As the effects of these policies cannot be distinguished from each other due to the data gap, we group them for the purpose of this analysis.

Italy

We used epidemiological data from the GitHub repository³⁸ maintained by the Italian Department of Civil Protection (Dipartimento della Protezione Civile). For policies, we primarily relied on the English version of the COVID-19 dossier “Chronology of main steps and legal acts taken by the Italian Government for the containment of the COVID-19 epidemiological emergency” written by the Dipartimento della Protezione Civile, ³⁹ and Wikipedia.⁴⁰

France

We used the region-level epidemiological dataset provided by France’s government website⁴¹ and supplemented it with numbers of confirmed cases by region on France’s public health website, which was previously updated daily through March 25.⁴² We obtained data on France’s policy response to the COVID-19 pandemic from the French government website, press releases from each regional public health site, ⁴³ and Wikipedia. ⁴⁴

United States

We used state-level epidemiological data from usafacts.org, ⁴⁵ which they compile from multiple sources. For policy responses, we relied on a number of sources, including the U.S. Centers for Disease Control (CDC), the National Governors Association, as well as various executive orders from county-and city-level governments, and press releases from media outlets.

Policy Data

Policies in administrative units were coded as binary variables, where the policy was coded as either 1 (after the date that the policy was implemented, and before it was removed) or 0 otherwise, for the affected administrative units. When a policy only affected a fraction of an administrative unit (e.g., half of the counties within a state), policy variables were weighted by the percentage of people within the administrative unit who were treated by the policy. We used the most recent population estimates we could find for countries’ administrative units (see the Population Data section in the Appendix). In order to standardize policy types across countries, we mapped each country-specific policy to one of the broader policy category variables in our analysis. In this exercise, we collected 168 policies for China, 59 for South Korea, 214 for Italy, 23 for Iran, 59 for France, and 1, 194 for the United States (see Supplementary Table 1). There are some cases where we encode policies that are necessarily in effect whenever another policy is in place, due in particular to the far-reaching implications of home isolation policies. In China, wherever home isolation is documented, we assume a local travel ban is enacted on the same day if we have not found an explicit local travel ban policy for a given locality. In France, we assume home isolation is accompanied by event cancellations, social distancing, and no-gathering policies; in Italy, we assume home isolation entails no-gathering, local travel ban, work from home, and social distancing policies; in the US, we assume shelter-in-place orders indicate that non-essential business closures, work from home policies, and no-gathering policies are in effect. For policy types that are enacted multiple times at increasing degrees of intensity within a locality, we add weights to the variable by escalating the intensity from 0 pre-policy in steps up to 1 for the final version of the policy (see the Policy Data section in the Appendix).

Epidemiological Data

We collected information on cumulative confirmed cases, cumulative recoveries, cumulative deaths, active cases, and any changes to domestic COVID-19 testing regimes, such as case definitions or testing methodology. For our regression analysis (Figure 2), we use active cases when they are available (for China and South Korea) and cumulative confirmed cases otherwise. We document quality control steps in the Appendix. Notably, for China and South Korea we acquired more granular data than the data hosted on the Johns Hopkins University (JHU) interactive dashboard;⁴⁸ we confirm that the number of confirmed cases closely match between the two data sources (see Extended Data Fig. 1). To conduct the econometric analysis, we merge the epidemiological and policy data to form a single data set for each country.

Econometric analysis

Reduced-Form Approach

The reduced-form econometric approach that we apply here is a “top down” approach that describes the behavior of aggregate outcomes y in data (here, infection rates). This approach can identify plausibly causal effects^{5, 14} induced by exogenous changes in independent policy variables z (e.g., school closure) without explicitly describing all underlying mechanisms that link z to y, without observing intermediary variables x (e.g., behavior) that might link z to y, or without other determinants of y unrelated to z (e.g., demographics), denoted w. Let f(·) describe a complex and unobserved process that generates infection rates y:

Process-based epidemiological models aim to capture elements of f(·) explicitly, and then simulate how changes in z, x, or w affect y. This approach is particularly important and useful in forward-looking simulations where future conditions are likely to be different than historical conditions. However, a challenge faced by this approach is that we may not know the full structure of f(·), for example if a pathogen is new and many key biological and societal parameters remain uncertain. Crucially, we may not know the effect that large-scale policy (z) will have on behavior (x(z)) or how this behavior change will affect infection rates (f(·)).

Alternatively, one can differentiate Equation 1 with respect to the k^th policy z_k: which describes how changes in the policy affects infections through all N potential pathways mediated by x₁, …, x_N. Usefully, for a fixed population observed over time, empirically estimating an average value of the local derivative on the left-hand-side in Equation 2 does not depend on explicit knowledge of w. If we can observe y and z directly and estimate changes over time : with data, then intermediate variables x also need not be observed nor modeled. The reduced-form econometric approach^{5, 14} thus attempts to measure directly, exploiting exogenous variation in policies z.

Model

Active infections grow exponentially during the initial phase of an epidemic, when the proportion of immune individuals in a population is near zero. Assuming a simple Susceptible-Infected-Recovered (SIR) disease model (e.g., ref. [¹¹]), the growth in infections during the early period is where I_t is the number of infected individuals at time t, β is the transmission rate (new infections per day per infected individual), γ is the removal rate (proportion of infected individuals recovering or dying each day) and S is the fraction of the population susceptible to the disease. The second equality holds in the limit S → 1, which describes the current conditions during the beginning of the COVID-19 pandemic. The solution to this ordinary differential equation is the exponential function where I_tl is the initial condition. Taking the natural logarithm and rearranging, we have

Anti-contagion policies are designed to alter g, through changes to β, by reducing contact between susceptible and infected individuals. Holding the time-step between observations fixed at one day (t₂ − t₁ = 1), we thus model g as a time-varying outcome that is a linear function of a time-varying policy where θ₀ is the average growth rate absent policy, policy_t is a binary variable describing whether a policy is deployed at time t, and θ is the average effect of the policy on growth rate g over all periods subsequent to the policy’s introduction, thereby encompassing any lagged effects of policies. ∊_t is a mean-zero disturbance term that captures inter-period changes not described by policy_t. Using this approach, infections each day are treated as the initial conditions for integrating Equation 4 through to the following day.

We compute the first differences log(I_t) − log(I_t−₁) using active infections where they are available, otherwise we use cumulative infections, noting that they are almost identical during this early period (except in China, where we use active infections). We then match these data to policy variables that we construct using the novel data sets we assemble and apply a reduced-form approach to estimate a version of Equation 6, although the actual expression has additional terms detailed below.

Estimation

To estimate a multi-variable version of Equation 6, we estimate a separate regression for each country c. Observations are for subnational units indexed by i observed for each day t. Because not all localities began testing for COVID-19 on the same date, these samples are unbalanced panels. To ensure data quality, we restrict our analysis to localities after they have reported at least ten cumulative infections.

A necessary condition for unbiased estimates is that the timing of policy deployment is independent of natural infection growth rates, ¹⁴ a mathematical condition that should be true in the context of a new epidemic. In established epidemiological models, including the standard SIR model above, early rates of infection within a susceptible population are characterized by constant exponential growth. This phenomenon is well understood theoretically, ¹³, ^{13, 27,46} has been repeatedly documented in past epidemics^{28, 29,47} as well as the current COVID-19 pandemic^11,, ¹² and implies constant infection growth rates in the absence of policy intervention. Thus, we treat changes in infection growth rates as conditionally independent of policy deployments since the correlation between a constant variable and any other variable is zero in expectation.

We estimate a multiple regression version of Equation 6 using ordinary least squares. We include a vector of subnational unit-fixed effects θ₀ (i.e., varying intercepts captured as coefficients to dummy variables) to account for all time-invariant factors that affect the local growth rate of infections, such as differences in demographics, socio-economic status, culture, and health systems.⁵ We include a vector of day-of-week-fixed effects δ to account for weekly patterns in the growth rate of infections that are common across locations within a country, however, in China, we omit day-of-week effects because we find no evidence they are present in the data – perhaps due to the fact that the outbreak of COVID-19 began during a national holiday and workers never returned to work. We also include a separate single-day dummy variable each time there is an abrupt change in the availability of COVID-19 testing or a change in the procedure to diagnose positive cases. Such changes generally manifest as a discontinuous jump in infections and a re-scaling of subsequent infection rates (e.g., See China in Figure 1), effects that are flexibly absorbed by a single-day dummy variable because the dependent variable is the first-difference of the logarithm of infections. We denote the vector of these testing dummies μ.

Lastly, we include a vector of P_c country-specific policy variables for each location and day. These policy variables take on values between zero and one (inclusive) where zero indicates no policy action and one indicates a policy is fully enacted. In cases where a policy variable captures the effects of collections of policies (e.g., museum closures and library closures), a policy variable is computed for each, then they are averaged, so the coefficient on this type of variable is interpreted as the effect if all policies in the collection are fully enacted. There are also instances where multiple policies are deployed on the same date in numerous locations, in which case we group policies that have similar objectives (e.g., suspension of transit and travel ban, or cancelling of events and no gathering) and keep other policies separate (i.e., business closure, school closure). The grouping of policies is useful for reducing the number of estimated parameters in our limited sample of data, allowing us to examine the impact of subsets of policies (e.g. Fig. 2c). However, policy grouping does not have a material impact on the estimated effect of all policies combined nor on the effect of actual policies, which we demonstrate by estimating a regression model where no policies are grouped and these values are recalculated (Supplementary Table 4, Extended Data Fig. 6).

In some cases (for Italy and the US), policy data is available at a more spatially granular level than infection data (e.g., city policies and state-level infections in the US). In these cases, we code binary policy variables at the more granular level and use population-weights to aggregate them to the level of the infection data. Thus, policy variables may take on continuous values between zero and one, with a value of one indicating that the policy is fully enacted for the entire population. Given the limited quantity of data currently available, we use a parsimonious model that assumes the effects of policies on infection growth rates are approximately linear and additively separable. However, future work that possesses more data may be able to identify important nonlinearities or interactions between policies.

For each country, our general multiple regression model is thus where observations are indexed by country c, subnational unit i, and day t. The parameters of interest are the country-by-policy specific coefficients θ_cp. We display the estimated residuals ∊_cit in Extended Data Fig. 10, which are mean zero but not strictly normal (normality is not a requirement of our modeling and inference strategy), and we estimate uncertainty over all parameters by calculating our standard errors robust to error clustering at the day level.¹⁴ This approach allows the covariance in ∊_cit across different locations within a country, observed on the same day, to be nonzero. Such clustering is important in this context because idiosyncratic events within a country, such as a holiday or a backlog in testing laboratories, could generate nonuniform country-wide changes in infection growth for individual days not explicitly captured in our model. Thus, this approach non-parametrically accounts for both arbitrary forms of spatial auto-correlation or systematic misreporting in regions of a country on any given day (we note that it generates larger estimates for uncertainty than clustering by i). When we report the effect of all policies combined (e.g., Figure 2b) we are reporting the sum of coefficient estimates for all policies , accounting for the covariance of errors in these estimates when computing the uncertainty of this sum.

Note that our estimates of θ and θ₀ in Equation 7 are robust to systematic under-reporting of infections, a major concern in the ongoing pandemic, due to the construction of our dependent variable. This remains true even if different localities have different rates of under-reporting, so long as the rate of under-reporting is relatively constant. To see this, note that if each locality i has a medical system that reports only a fraction ψi of infections such that we observe Ĩ_it = ψ_iI_it rather an actual infections I_it, then the left-hand-side of Equation 7 will be and is therefore unaffected by location-specific and time-invariant under-reporting. Thus systematic under-reporting does not affect our estimates for the effects of policy θ. As discussed above, potential biases associated with non-systematic under-reporting resulting from documented changes in testing regimes over space and time are absorbed by region-day specific dummies μ.

However, if the rate of under-reporting within a locality is changing day-to-day, this could bias infection growth rates. We estimate the magnitude of this bias (see Extended Data Fig. 2), and verify that it is quantitatively small. Specifically, if Ĩ_it = ψ_itI_it where ψ_it changes day-to-day, then where log(ψ_it)−log(ψ_ijt−₁) is the day-over-day growth rate of the case-detection probability. Disease surveillance has evolved slowly in some locations as governments gradually expand testing, which would cause ψ_it to change over time, but these changes in testing capacity do not appear to significantly alter our estimates of infection growth rates. In Extended Data Fig. 2, we show one set of epidemiological estimates²³ for log(ψ_it) − log(ψ_{i, t−}₁). Despite random day-to-day variations, which do not cause systematic biases in our point estimates, the mean of log(ψ_it) − log(ψ_{i, t}₋₁) is consistently small across the different countries: 0.05 in China, 0.064 in Iran, 0.019 in South Korea, −0.058 in France, 0.031 in Italy, and 0.049 in the US. The average of these estimates is 0.026, potentially accounting for 7.3% of our global average estimate for the no-policy infection growth rate (0.36). These estimates of log(ψ_it) − log(ψ_{i, t−}₁) also do not display strong temporal trends, alleviating concerns that time-varying under-reporting generates sizable biases in our estimated effects of anti-contagion policies.

Transient dynamics

In China, we are able to examine the transient response of infection growth rates following policy deployment because only three policies were deployed early in a seven-week sample period during which we observe many cities simultaneously. This provides us with sufficient data to estimate the temporal structure of policy effects without imposing assumptions regarding this structure. To do this, we estimate a distributed-lag model that encodes policy parameters using weekly lags based on the date that each policy is first implemented in locality i. This means the effect of a policy implemented one week ago is allowed to differ arbitrarily from the effect of that same policy in the following week, etc. These effects are then estimated simultaneously and are displayed in Fig. 2 (also Supplementary Table 3). Such a distributed lag approach did not provide statistically meaningful insight in other countries using currently available data because there were fewer administrative units and shorter periods of observation (i.e. smaller samples), and more policies (i.e. more parameters to estimate) in all other countries. Future work may be able to successfully explore these dynamics outside of China.

As a robustness check, we examine whether excluding the transient response from the estimated effects of policy substantially alters our results. We do this by estimating a “fixed lag” model, where we assume that policies cannot influence infection growth rates for L days, recoding a policy variable at time t as zero if a policy was implemented fewer than L days before t. We re-estimate Equation 7 for each value of L and present results in Extended Data Fig. 5 and Supplementary Table 5.

Alternative disease models

Our main empirical specification is motivated with an SIR model of disease contagion, which assumes zero latent period between exposure to COVID-19 and infectiousness. If we relax this assumption to allow for a latent period of infection, as in a Susceptible-Exposed-Infected-Recovered (SEIR) model, the growth of the outbreak is only asymptotically exponential.¹¹ Nonetheless, we demonstrate that SEIR dynamics have only a minor potential impact on the coefficients recovered by using our empirical approach in this context. In Extended Data Figs. 8 and 9 we present results from a simulation exercise which uses Equations 9-11, along with a generalization to the SEIR model¹¹ to generate synthetic outbreaks (see Supplementary Methods Section 2). We use these simulated data to test the ability of our statistical model (Equation 7) to recover both the unimpeded growth rate (Extended Data Fig. 8) as well as the impact of simulated policies on growth rates (Extended Data Fig. 9) when applied to data generated by SIR or SEIR dynamics over a wide range of epidemiological conditions.

Projections

Daily growth rates of infections

To estimate the instantaneous daily growth rate of infections if policies were removed, we obtain fitted values from Equation 7 and compute a predicted value for the dependent variable when all P_c policy variables are set to zero. Thus, these estimated growth rates capture the effect of all locality-specific factors on the growth rate of infections (e.g., demographics), day-of-week-effects, and adjustments based on the way in which infection cases are reported. This counterfactual does not account for changes in information that are triggered by policy deployment, since those should be considered a pathway through which policies affect outcomes, as discussed in the main text. Additionally, the “no-policy” counterfactual does not model previously unobserved changes in behavior that might occur if fundamentally new behaviors emerge even in the absence of government intervention. When we report an average no-policy growth rate of infections (Figure 2a), it is the average value of these predictions for all observations in the original sample. Location-and-day specific counterfactual predictions (), accounting for the covariance of errors in estimated parameters, are shown as red markers in Figure 3.

Cumulative infections

To provide a sense of scale for the estimated cumulative benefits of effects shown in Figure 3, we link our reduced-form empirical estimates to the key structures in a simple SIR system and simulate this dynamical system over the course of our sample. The system is defined as the following: where S_t is the susceptible population and R_t is the removed population. Here β_t is a time-evolving parameter, determined via our empirical estimates as described below. Accounting for changes in S becomes increasingly important as the size of cumulative infections (I_t + R_t) becomes a substantial fraction of the local subnational population, which occurs in some no-policy scenarios. Our reduced-form analysis provides estimates for the growth rate of active infections () for each locality and day, in a regime where S_t ≈ 1. Thus we know but we do not know the values of either of the two right-hand-side terms, which are required to simulate Equations 9-11. To estimate γ, we note that the left-hand-side term of Equation 11 is which we can observe in our data for China and South Korea. Computing first differences in these two variables (to differentiate with respect to time), summing them, and then dividing by active cases gives us estimates of γ (medians: China=0.11, Korea=0.05). These values differ slightly from the classical SIR interpretation of γ because in the public data we are able to obtain, individuals are coded as “recovered” when they no longer test positive for COVID-19, whereas in the classical SIR model this occurs when they are no longer infectious. We adopt the average of these two medians, setting γ = .08. We use medians rather than simple averages because low values for I induce a long right-tail in daily estimates of γ and medians are less vulnerable to this distortion. We then use our empirically-based reduced-form estimates of ĝ (both with and without policy) combined with Equations 9-11 to project total cumulative cases in all countries, shown in Figure 4. We simulate infections and cases for each administrative unit in our sample beginning on the first day for which we observe 10 or more cases (for that unit) using a time-step of 4 hours. Because we observe confirmed cases rather than total infections, we seed each simulation by adjusting observed I_t on the first day using country-specific estimates of case detection rates. We adjust existing estimates of case under-reporting²³ to further account for asymptomatic infections assuming an infection-fatality ratio of 0.075%.²⁵ We assume R_t = 0 on the first day. To maintain consistency with the reported data, we report our output in confirmed cases by multiplying our simulated I_t + R_t values by the aforementioned proportion of infections confirmed. We estimate uncertainty by resampling from the estimated variance-covariance matrix of all regression parameters. In Extended Data Fig. 7, we show sensitivity of this simulation to the estimated value of γ as well as to the use of a Susceptible-Exposed-Infected-Recovered (SEIR) framework. In Supplementary Table 6, we show sensitivity of this simulation to the assumed infection-fatality ratio (see Supplementary Methods Section 1).

Data and Code Availability

The datasets generated during and/or analysed during the current study are available at https://github.com/bolliger32/gpl-covid (DOI: 10.5281/zenodo.3832367). For easier replication, we have also created a CodeOcean “capsule” - which contains a pre-built computing environment in addition to the source code and data. This is available at https://codeocean.com/capsule/1887579/tree/v1 (DOI: I0.24433/C0.6625287.v2). Future updates and/or extensions to data or code will be listed at http://www.globalpolicy.science/covid19.

End Notes

Author Contributions

SH conceived of and led the study. All authors designed analysis, interpreted results, designed figures, and wrote the paper. China: LYH, TW collected health data, LYH, TW, JT collected policy data, LYH cleaned data. South Korea: JL Collected health data, TC, JL collected policy data, TC cleaned data. Italy: DA collected health data, PL collected policy data, DA cleaned data. France: SAP collected health data, SAP, JT, HD collected policy data, SAP cleaned data. Iran: AH collected health data and policy data, AH, DA cleaned data. USA: ER, KB collected health data, EK collected policy data, ER, DA, KB cleaned data. IB collected geographic and population data for all countries. SH designed the econometric model. SH, SAP, JT conducted econometric analysis for all countries. KB, IB, AH, ER, EK designed and implemented epidemiological models and projections. SAP, KB, IB, JT, AH, EK designed and implemented robustness checks. HD created Fig. 1, TC created Fig. 2, JT created Fig. 3, ER created Fig. 4, DA created SI Table 1, LYH, JL created SI Table 2, JT created SI Table 3, JT created SI Table 4, SAP, JT created SI Table 5, KB created SI Table 6, LYH created ED Figs. 1-2, SAP created ED Figs. 3-5, JT created ED Fig. 6, KB created ED Fig. 7, IB created ED Figs. 8-9, JT created ED Fig. 10. DA, IB, PL managed policy data collection and quality control. IB, TC managed the code repository. IB, PL ran project management. EK, TW, JT, PL managed literature review. LYH, EK, TW managed References. PL managed the Extended Data and Appendix.

Ethics Declaration

The authors declare no conflicts of interest.

Additional Information

Supplementary Information is available for this paper. Correspondence and requests for materials should be addressed to Solomon Hsiang (shsiang{at}berkeley.edu).

Extended Data for The Effect of Large-Scale Anti-Contagion Policies on the COVID-19 Pandemic”

All data and code are available at https://github.com/bolliger32/gpl-covid.

Additional resources and updates are available at www.globalpolicy.science/covid19.

For repository related questions, please contact bolliger{at}berkeley.edu.

For data related questions, please contact peiley{at}berkeley.edu.

This document last updated: May 16, 2020. Data last updated: April 6, 2020.

Extended Data Figure 1 Validating disaggregated epidemiological against aggregated data from the Johns Hopkins Center for Systems Science and Engineering.

Comparison of cumulative confirmed cases from a subset of regions in our collated epidemiological dataset to the same statistics from the 2019 Novel Coronavirus COVID-19 (2019-nCoV) Data Repository by the Johns Hopkins Center for Systems Science and Engineering (JHU CSSE).¹ We conduct this comparison for Chinese provinces and South Korea, where the data we collect are from local administrative units that are more spatially granular than the data in the JHU CSSE database. a, In China, we aggregate our city-level data to the province level, and b, in Korea we aggregate province-level data up to the country level. Small discrepancies, especially in later periods of the outbreak, are generally due to imported cases (international or domestic) that are present in national statistics but which we do not assign to particular cities (in China) or provinces (in Korea).

Extended Data Figure 2 Estimated trends in case detection over time within each country.

Systematic trends in case detection may potentially bias estimates of no-policy infection growth rates (see Equation 8). We estimate the potential magnitude of this bias using data from the Centre for Mathematical Modelling of Infectious Diseases.² Markers indicate daily first-differences in the logarithm of the fraction of estimated symptomatic cases reported for each country over time. The average value over time (solid line and value denoted in panel title) is the average growth rate of case detection, equal to the magnitude of the potential bias. For example, in the main text we estimate that the infection growth rate in the United States is 0.29 (Figure 2A), of which growth in case detection might contribute 0.049 (this figure). Sample sizes are 75 in China, 41 in Iran, 40 in South Korea, 29 in France, 40 in Italy, and 32 in the US.

Extended Data Figure 3 Robustness of the estimated no-policy growth rate of infections and the combined effect of policies to withholding blocks of data from entire regions.

For each country, we re-estimated Eq. 7 using real data k times, each time withholding one of the k first-level administrative regions (“Adm1,” i.e. state or province) in that country. Each gray circle is either (a) the estimated no-policy growth rate or (b) the total effect of all policies combined, from one of these k regressions. Red and blue circles show estimates from the full sample, identical to results presented in panels A and B of Figure 2, respectively. For each country panel, if a single region is influential, the estimated value when it is withheld from the sample will appear as an outlier. Some regions that appear influential are highlighted with an open pink circle. As in Figure 2B of the main text, we estimate a distributed lag model for China and display each of the estimated weekly lag effects (red circle is the same “without Hubei” sample for lags). The full sample includes 3, 684 observations in China, 595 in South Korea, 2, 898 in Italy, 548 in Iran, 270 in France, and 1, 238 in the US.

Extended Data Figure 4 Robustness of the estimated effects of individual policies to withholding blocks of data from entire regions.

Same as Extended Data Figure 3, but for individual policies (analogous to Figure 2C in the main text). In cases where two regions are influential, a second region is highlighted with an open green circle. The full sample includes 3, 669 observations in China, 595 in South Korea, 2, 898 in Italy, 548 in Iran, 270 in France, and 1, 238 in the US.

Extended Data Figure 5 Evidence supporting models where policies affect infection growth rates in the days following deployment.

Existing evidence has not demonstrated whether policies should affect infection growth rates in the days immediately following deployment. It is therefore not clear ex ante whether the policy variables in Eq. 7 should be encoded as “on” immediately following a policy deployment. We estimate “fixed-lag” models in which a fixed delay between a policy’s deployment and its effect is assumed (see Supplementary Methods Section 3). If a delay model is more consistent with real world infection dynamics, these fixed lag models should recover larger estimates for the impact of policies and exhibit better model fit. a, R-squared values associated with fixed-lag lengths varying from zero to fifteen days. Center values represent the R squared value in our sample, whiskers are 95% CI computed through resampling with replacement. In-sample fit generally declines or remains unchanged if policies are assumed to have a delay longer than four days. b, Estimated effects for no lag (the model reported in the main text) and for fixed-lags between one and five days. Center values represent the point estimate, error bars are 95% CI. Estimates generally are unchanged or shrink towards zero (e.g. Home isolation in Iran), consistent with mis-coding of post-policy days as no-policy days. The sample size is 595 in South Korea, 2, 898 in Italy, 548 in Iran, 270 in France, and 1, 238 in the US.

Extended Data Figure 6 Estimated infection or hospitalization growth rates with actual anti-contagion policies and in a “no policy” counterfactual scenario.

a, The estimated daily growth rates of active (China, South Korea) or cumulative (all others) infections based on the observed timing of all policy deployments within each subnational unit (blue) and in a scenario where no policies were deployed (red). Identical to Figure 3 in the main text, but using an alternative disaggregated encoding of policies that does not group any policies into policy packages. The sample size is 3, 669 in China, 595 in South Korea, 2, 898 in Italy, 548 in Iran, 270 in France, and 1, 238 in the US. b, Same as Figure 3 in the main text, but Eq. 7 is implemented for a single example administrative unit, Wuhan, China. The sample size is 46 observations. c, Same as Figure 3 in the main text, but using hospitalization data from France rather than cumulative cases (the French government stopped reporting cumulative cases after March 25, 2020). The sample size is 424 observations. For all panels, the difference between the with- and no-policy predictions is our estimated effect of actual anti-contagion policies on the growth rate of infections (or hospitalizations). The markers are daily estimates for each subnational administrative unit (vertical lines are 95% confidence intervals). Black circles are observed changes in log(infections) (or diamonds for log(hospitalizations)), averaged across observed administrative units.

Extended Data Figure 7 Sensitivity of estimated averted/delayed infections to the choice of γ and σ in an SIR/SEIR framework.

This figure displays the sensitivity of total averted/delayed cases presented in Figure 4 of the main text to alternative modeling assumptions. We compute total cases across the respective final days in our samples for the six countries presented in our analysis. The figure displays how these totals vary with eight values of γ (0.05-0.4) and four values of σ (0.2, 0.33, 0.5, ∞), where the final value of σ (∞) corresponds to the SIR model. a, The simulated total number of infections under no policy. b, Same, but using actual policies. c, The difference between (a) and (b), which is the total number of averted/delayed infections. d, Same as (c), but on a logarithmic scale similar to Figure 4 in the main text (a - c are on a linear scale, trimmed to show details). Figure 4 in the main text uses γ = 0.079, which we calculate using empirical recovery/death rates in countries where we observe them (China and South Korea, see Methods). If we assume a 14-day delay between infected individuals becoming non-infectious and being reported as “recovered” in the data, we would calculate γ = 0.18. Figure 4 in the main text assumes σ = ∞.

Extended Data Figure 8 Simulating reduced-form estimates for the no-policy growth rate of infections for different population regimes and disease dynamics.

We examine the performance of reduced-form econometric estimators through simulations in which different underlying disease dynamics are assumed (see Supplementary Information Section 3). Each histogram shows the distribution of econometrically estimated values across 1, 000 simulated outbreaks. Estimates are for the no-policy infection growth rate (analogous to Figure 2a) when three different policies are deployed at random moments in time. The black line shows the correct value imposed on the simulation and the red histogram shows the distribution of estimates using the regression in Eq. 7, applied to data output from the simulation. The grey dashed line shows the mean of this distribution. The 12 subpanels describe the results when various values are assigned to the mean infectious period (γ⁻¹) and mean latency period (σ⁻¹) of the disease. “σ = ∞” is equivalent to SIR disease dynamics. In each panel, S_min is the minimum susceptible fraction observed across all 1, 000 45-day simulations shown in each panel. In the real datasets used in the main text, after correcting for country-specific under-reporting, S_min across all units analyzed is 0.72 and 95% of the analyzed units finish with S_min > 0.91. “Bias” refers to the distance between the dashed grey and black line as a percentage of the true value. a, Simulations in near-ideal data conditions in which we observe active infections within a large population (such that the susceptible fraction of the population remains high during the sample period, similar to those in our data for Chongqing, China). b, Simulations in a non-ideal data scenario where we are only able to observe cumulative infections in a small population (similar to those in our sample for Cremona, Italy).

Extended Data Figure 9 Simulating reduced form estimates for anti-contagion policy effects for different population regimes and assumed disease dynamics.

Same as Extended Data Figure 8, but estimates are for the combined effect of three different policies (analogous to Figure 2b) that are deployed at random moments in time.

Extended Data Figure 10 Regression residuals for the growth rates of COVID-19 by country.

These plots show the estimated residuals from Equation 7 for each country-specific econometric model. Histograms (left) show the estimated unconditional probability density function. Quantile plots (right) show quantiles of the cumulative density function (y-axis) plotted against the same quantiles for a Normal Distribution. For additional details, see Fig. 3 and the Econometric Analysis section of Methods.

Supplementary Information for “The Effect of Large-Scale Anti-Contagion Policies on the COVID-19 Pandemic”

The following sections provide additional information to support the article and to document our data collection and processing procedures.

All data and code are available at https://github.com/bolliger32/gpl-covid.

Additional resources and updates are available at www.globalpolicy.science/covid19.

For repository related questions, please contact bolliger{at}berkeley.edu.

For data related questions, please contact peiley{at}berkeley.edu.

This document last updated: May 16, 2020. Data last updated: April 6, 2020.

The Supplementary Information contains three sections: Supplementary Notes, Supplementary Methods, and Supplementary Tables.

The Supplementary Notes section details the basic accounting of policy deployment decisions in each of the six countries we analyze, and describes the data acquisition and processing procedure for the epidemiological and policy data used in this paper. The sources for both types of data come from a variety of in-country data sources, which include government public health websites, regional newspaper articles, and Wikipedia crowd-sourced information. We have supplemented this data with international data compilations. A list of the epidemiological and policy data compiled for this analysis can be found here.

The Supplementary Methods section describes sensitivity analyses and simulations performed to verify the robustness of our model, including: the sensitivity of our counterfactual projections to varying epidemiological parameters; and the sensitivity of our regression estimates to alternative lag structures, withholding of data, and differing policy groupings.

The Supplementary Tables section contains tables detailing:

Number of unique anti-contagion policies tabulated by administrative division of each country.
Epidemiological data in Wuhan prior to policy intervention, and estimates of the initial infection growth rate and case doubling times.
Full regression results from our main regression estimating the effect of policy on growth rates.
Estimates of the effect of policies on infection growth rate using a more disaggregated grouping of policies.
Estimates of the effect of policies on infection growth rate assuming a variety of lag structures.
Estimates of the cumulative number of confirmed cases averted assuming a variety of country-specific underreporting ratios.

Supplementary Notes

Chronology of Policy Implementation

In the main article, we analyze how infection growth rates change when anti-contagion policies are deployed. In order for these estimated associations to be interpreted as plausibly causal, it must be the case that changes in policy are not correlated with changes in infection growth rates that would have occured in the absence of policy actions. For locations and periods where policy¹ was actually deployed, it is not possible to directly observe how growth rates would have changed in the absence of policy actions.² In general, it is well-established that in the absence of policy, early rates of infection within a susceptible population follow almost-perfect exponential growth.^{3, 4, 5, 6, 7, 8, 9, 10} This fact would suggest that in the absence of policy, infection growth rates would be constant^{11, 12} and, therefore, changes in the infection growth rate could not be correlated with the timing of policy deployments (since correlation with a constant variable is zero). However, it is nonetheless also worthwhile to consider how the timing of policy actions were determined in practice. In this section, we provide a basic accounting of policy deployment decisions in each of the six countries we analyze, focusing on the timing of major events and reported motivations for notable policy deployments. These accounts are intended to familiarize readers with the general decision-making context of each country, although they are not exhaustive and are not intended to capture the full complexity of decision-making in the ongoing pandemic.

As stated in the main text, policies were generally not initially deployed with any reference to observed or anticipated natural changes in growth rates. In general, policies tended to be deployed in response to high total numbers of cases or to outbreaks in other regions, after long delays due to political constraints, and often with timing that coincided with arbitrary events, like the start and end of the week, or holidays. For example, epidemic planning in France explicitly tied policy actions to case-counts in specific regions, and epidemiological guidance in the US explicitly recommended policy actions depending on when thresholds in total case counts were passed. In South Korea, policy actions across the country were initially triggered by an idiosyncratic outbreak in a Shincheonji Church in Daegu, and in Iran they were triggered by an outbreak in Qom coinciding with a religious pilgrimage. In China and Italy, nation-wide policy actions were deployed relatively swiftly by the central government in response to a regional outbreak, with limited variation across locations that could be correlated with changes in local growth rates. In China, Italy, and the US, policies tended to be deployed during the weekend or on a Monday or Friday, with the preferred day of the week varying across these countries.