Abstract
We evaluate the impact of government-mandated proof of vaccination requirements for access to public venues and non-essential businesses on COVID-19 vaccine uptake. We find that the announcement of a mandate is associated with a rapid and significant surge in new vaccinations (more than 60% increase in weekly first doses), using the variation in the timing of these measures across Canadian provinces in a difference-in-differences approach. Time-series analysis for each province and for France, Italy and Germany corroborates this finding. Counterfactual simulations using our estimates suggest the following cumulative gains in the vaccination rate among the eligible population (age 12 and over) as of October 31, 2021: up to 5 percentage points (p.p.) (90% CI 3.9–5.8) for Canadian provinces, adding up to 979,000 (425,000-1,266,000) first doses in total for Canada (5 to 13 weeks after the provincial mandate announcements), 8 p.p. (4.3–11) for France (16 weeks post-announcement), 12 p.p. (5–15) for Italy (14 weeks post-announcement) and 4.7 p.p. (4.1–5.1) for Germany (11 weeks post-announcement).
Introduction
Immunization has proven very effective for reducing the spread and severity of COVID-19, with large reductions in the risk of severe outcomes for vaccinated people, [1–4]. Yet, following a rapid uptake in early 2021, vaccinations in many countries (see Extended Data Fig. 1) slowed down significantly in the summer months. In addition, even locations with high vaccination rates experienced increased viral transmission or had to maintain or re-introduce non-pharmaceutical interventions such as mask wearing or indoor capacity limits in fall 2021 because of the elevated reproduction rate of the Delta variant.
Achieving high COVID-19 vaccination coverage is therefore essential for reducing the health and economic impacts of the epidemic. Moreover, booster doses or vaccines with updated formulations may be necessary in the face of new variants. Public health authorities throughout the world have sought effective strategies to increase vaccine uptake, especially among the hesitant or procrastinating.
In response to this challenge, various local or national governments have introduced proof of vaccination mandates or certificates, [5–7], which allow vaccinated persons to attend non-essential sports or social settings and events such as concerts, stadiums, museums, restaurants, bars, etc. The goal of these policies is twofold: to provide incentives for immunization and to reduce viral transmission in risky indoor or crowded settings.
We evaluate and quantify the effect of proof of vaccination mandates on first-dose vaccine uptake in the ten Canadian provinces and three European countries (France, Italy and Germany). Some mandates accept a recent negative or past positive COVID test result as a substitute of vaccination or allow businesses to opt out if they abide by additional restrictions. Among the jurisdictions we consider, the mandates in France, Italy, Germany, Alberta and Saskatchewan allowed such options during the studied period. We chose these locations since they have similar economies, demographics and vaccine access, and all announced and implemented mandates in July–October 2021 (see Supplementary Table 1), a period with minimal binding vaccine supply or access constraints and a high base first-dose vaccination rate (above 60% of those eligible in the three countries and above 80% in Canada at the time of the mandate announcements, see Supplementary Table 2). Hence, we evaluate the mandates’ impact on people, such as the vaccine hesitant, that have remained unvaccinated for weeks or months after immunization was available to them (see Supplementary Table 3).
While, given past evidence from HPV, Tdap or hepatitis A immunizations [8–10], requiring proof of vaccination is expected to raise vaccine uptake, the magnitude and speed of the increase are hard to predict: they depend on the relative importance of the factors leading to delay or hesitancy, e.g., lack of social or economic incentives, misinformation, or entrenched political or religious beliefs. We use first doses as the main outcome in our statistical analysis because they most directly reflect the decision to be immunized.
Fig. 1 plots the weekly vaccine first doses administered in the four most populous Canadian provinces and four European countries. All except Spain introduced a province-or country-wide mandate in the studied period. We observe a sizable boost in vaccine uptake after the mandate announcement (the dashed vertical line) in all four provinces and in France, Italy and Germany, often in contrast to a sharp decline in the pre-announcement weeks. In France, daily first-dose appointments also show a striking surge on the day after the mandate announcement (see Extended Data Fig. 2). In contrast, Spain exhibits a steady decrease in weekly first doses over the displayed period.
Motivated by this evidence, we address two important policy-relevant questions. We first estimate the magnitude of the increase in first dose vaccinations after a mandate announcement, controlling for other possible factors. Second, we evaluate how long these vaccination gains persist and the cumulative effect on vaccine uptake. We do not address ethical considerations in this paper, [11]. Our goal is to assess the mandates’ effectiveness purely in terms of raising vaccine uptake, which can then be weighed against various political or enforcement costs and compared to other policies, such as financial incentives (cash, gift cards, lotteries), [12–17], or behavioural nudges (e.g., messages from experts, appointment reminders), [12, 18–20], for which mixed or negative results have been reported.
To quantify the mandates’ impact on vaccine uptake, we use Canadian provincial data in a difference-in-differences (DID) identification strategy based on the time variation in mandate announcement dates across different geographic units in the same country, Canada (from August 5, 2021 in Que-bec to September 21, 2021 in Prince Edward Island). In Canada, the provinces are separate jurisdictions with extensive powers over health policy, while the vaccines are procured by the federal government and allocated in proportion to provincial population. In contrast, the French, Italian or German mandates apply at the national level, which makes it more challenging to separate their effect from that of time trends or other concurrent events or policies.
We estimate a behavioural model in which the decision to receive a COVID-19 vaccine, measured by new weekly first doses, is affected by the policy setting (whether a proof of vaccination mandate has been announced) and the current COVID-19 epidemiological and public health conditions (‘information’), proxied by weekly cases and deaths, [21, 22]. We control for other potential confounding variables and unobserved heterogeneity with time and location (i.e., province) fixed effects (see Methods for details).
We complement and extend the DID results for Canada with a structural-break and time-series analysis, which allows us to study the mandates’ impact on vaccine uptake over a longer period. We obtain individual policy impact estimates for each Canadian province and for France, Italy and Germany, and identify potential factors contributing to the heterogeneity of the estimates, including the time between the mandate announcement and implementation and the fraction of the population that is already vaccinated at the time of announcement. We then use the time-series estimates in counterfactual simulations and compute the cumulative gains in vaccine uptake following the mandates over our study period ending October 31, 2021, relative to a hypothetical scenario in absence of mandates.
Results
Fig. 2 plots the raw-data time profile of weekly first doses after a proof of vaccination mandate announcement, with doses in the week ending on the announcement date normalized to 100. The figure shows that weekly first doses in the Canadian provinces and in France, Italy and Germany grow quickly, reach a peak one to five weeks after the announcement date and then decrease, as in the pre-announcement trend in most locations (see Extended Data Fig. 3). New Brunswick, Newfoundland, Alberta, Nova Scotia, Saskatchewan and France registered increases in vaccine uptake of over 100% relative to the pre-announcement week. In other locations, e.g., Quebec, Ontario, Manitoba and Germany the observed increase is more moderate, less than 50%.
Canadian provinces – difference-in-differences analysis
We use a difference-in-differences (DID) statistical method, the Sun and Abraham [23] treatment effect heterogeneity robust estimator (see Methods), to study the average effect of proof of vaccination mandates in Canada.
In Fig. 3, we plot results from an event study analysis of weekly first doses administered in the ten Canadian provinces, from six weeks prior to five or more weeks after the announcement of a mandate. We use as control group (latest treated) the last five provinces to announce a mandate (Alberta, New Brunswick, Saskatchewan, Newfoundland and Prince Edward Island). We chose this control group as the latest in time representative set of provinces containing a mix of smaller and larger provinces by population. This choice implies June 15, 2021 to September 14, 2021 (the day before mandate announcement in Alberta and New Brunswick, see Supplementary Table 1) as our baseline DID time period. We present results with different control groups corresponding to earlier or later sample end dates in Supplementary Table 4.
Fig. 3 shows a lack of mandate-associated pre-trend in the data – the DID estimates before the mandate announcement are statistically indistinguishable from zero. This addresses the potential endogeneity concern (parallel trends assumption) that provinces that announced a mandate may have had a different trend in first-dose vaccinations than provinces that did not announce a mandate. Second, the impact of the mandate announcement on first-dose vaccine uptake is realized relatively quickly and is large in magnitude — an increase of 42% (35 log points, p = 0.015, 95% CI 9–62) in new weekly doses in the first post-announcement week and 71% (54 log points, p = 0.001, 95% CI 21–86) in the second week, each relative to one week before the announcement. The observed quick increase in uptake mitigates possible concerns that vaccine supply or scheduling constraints may be affecting our results. Third, the policy effect persists over the 6-week post-announcement period we analyze (T = 0 to T = 5). Unfortunately, data limitations (the timing of the announcements) and the need for a not-yet-treated control group in the DID method do not allow us to investigate longer horizons.
Table 1 displays DID estimates of the mandate effect on first-dose uptake, relative to the pre-mandate period, controlling for information (cases and deaths) and time and location fixed effects. Columns (1) and (2) show that a mandate announcement is associated with an average increase of about 66 percent (50.6 log points, p = 0.001, 95% CI 25-77 in column 2) in weekly first doses. In column (3), we report estimates where the mandate policy variable is decomposed into six binary variables (one for each week after the announcement date) to account for dynamic effects. The results indicate a sharp increase in weekly first doses of 43 percent (36 log points, p = 0.005, 95% CI 14–58) in the week beginning at the announcement date (week 0). The increase is sustained throughout the post-announcement sample period, as all estimates in column (3) are positive (p < 0.06) and larger than the week 0 estimate. The DID analysis thus yields no evidence that short-term intertemporal substitution (which would be manifested as negative estimates in the later weeks) is the mechanism behind the observed boost in first-dose vaccinations in the studied period.
We perform a range of robustness checks and sensitivity analysis on our main DID results. Our result of a more than 60% increase in weekly first doses on average over the post-announcement weeks remains robust when using alternative dates and treatment groups varying from the first three to the first nine provinces to announce a mandate (see Supplementary Table 4 and Extended Data Fig. 5). We also allow for alternative initial sample dates or lags in the policy or information variables (see Extended Data Fig. 5). Additional robustness checks include using different control variables (hospitalizations), population weights, OLS two-way fixed effects, daily data, levels data, alternative ways to compute the standard errors, and using randomized (placebo) announcement dates, see Extended Data Fig. 6 and Supplementary Tables 5 and 6 for details. A possible concern is that case or death counts may be correlated with the mandate announcements (while also affecting vaccination rates as information, for which we control). We ran event-study analyses analogous to that in Fig. 3 but using weekly cases or deaths as the outcome and find that the estimates before the announcements are statistically indistinguishable from zero. This suggests an absence of significant differential pre-trends in cases or deaths across the treated and control provinces.
Most provincial proof of vaccination mandates required two doses to be considered adequately vaccinated with a vaccine offered in Canada during the study period: Pfizer (Comirnaty), Moderna (SpikeVax) or AstraZeneca (Vaxzevria); there were limited exceptions in Quebec (a prior infection could count as first dose) and British Columbia (only one dose was required between Sep. 13 and Oct. 23, 2021). In Supplementary Table 7, we use log of weekly second doses as the outcome. We do not find a statistically significant effect of mandate announcements on second dose uptake in our sample period, consistent with second doses being more evenly spread out over time. Extended Data Fig. 7 also shows that, unlike the sharp boost in first doses, there are only small or gradual post-mandate increases in second doses in some of the provinces. One possible explanation for the lack of large increases in second doses a few weeks after the spikes in first doses is that mandates may have encouraged some people that already had their first doses to obtain their second dose sooner, thus shifting some second doses forward and dampening the lagged effect.
We also do not find statistically significant effect on first-dose uptake associated with the mandates’ implementation dates additional to the announcement date effect; see Supplementary Information for details.
Time-series analysis
We complement and extend the DID panel-data results with a structural break and time-series analysis using the ‘interrupted time-series analysis’ (ITSA) method [24], which models the relationship between the outcome (weekly first doses) and the policy variable (mandate announcement) and controls for time trends, lagged outcomes and information (weekly cases and deaths), see Methods for details. This allows us to estimate the mandates’ longer-term impact on vaccine uptake separately for each of the ten Canadian provinces, as well as for France, Italy and Germany, using all the data up to October 31, 2021.
We first test for the presence of a structural break at the announcement date and find that we can reject the null hypothesis of no break for each of the provinces and countries; see Supplementary Table 8 and Methods for details. In Table 2, we report the time-series estimates for each country and province in our data. In columns (1)-(3), we report the estimated coefficients on the mandate announcement variable, and on two time trends: a linear daily time trend (estimate ) and an ‘interaction’ time trend reflecting the postmandate trend slope change (estimate ). Since the outcome and information variables are level variables, in column (4), we use the augmented Dickey-Fuller (ADF) unit root test to test whether the residuals are stationary, to avoid a spurious regression. We reject the null hypothesis that the residual contains a unit root at the 95% confidence level for all locations except Newfoundland, where we can reject the null at the 90% level.
The estimates, for the initial rise in weekly first doses after mandate announcement in Table 2 are large and statistically significantly positive for all three countries: 17% (15.4 log points, p = 0.004, 95% CI 5.0–25.8) for France, 179% (103 log points, p < 0.001, 95% CI 53–152) for Italy and 55% for Germany (43.8 log points, p < 0.001, 95% CI 31.4–56.1), relative to the respective pre-mandate trends. A comparison of Fig. 1 and Extended Data Fig. 2 suggests that the relatively low French estimate may reflect a lag between appointment booking and vaccine administration. We also obtain statistically significantly positive and large estimates of the increase in weekly first doses after mandate announcement, , for all provinces except Quebec. The estimate for Quebec is 0.389 (p < 0.001, 95% CI 0.265–0.514) and statistically significantly different from zero when we use deaths and hospitalizations as information; see Supplementary Table 9. The estimated mandate effect varies across the provinces: e.g., a 34% (29 log points, p = 0.007, 95% CI 8.2–50.5) initial increase in Ontario vs. 326% (145 log points, p < 0.001, 95% CI 110– 181) in Alberta. The range of the provincial estimates is consistent with our DID estimate in Table 1 for the average mandate effect for Canada; however, the two estimation methods are not directly comparable.
The estimated mandate effect on vaccine uptake tends to be larger for provinces experiencing a surge in cases at the same time (for which we control), namely Alberta, Saskatchewan and New Brunswick; see Extended Data Fig. 3. These provinces also announced their mandates relatively late (mid-September) and set a relatively short time interval between mandate announcement and implementation (see Supplementary Table 1). A larger fraction of eligible unvaccinated people at the time of mandate announcement is also positively associated with a larger policy effect estimate. We illustrate these associations in Extended Data Fig. 8.
The baseline time trend in first doses is downward sloping in all countries and provinces (although not statistically significant in the two smallest provinces, Newfoundland and Prince Edward Island) and indicates a steady decrease in new vaccinations of 0.8% to 3.4% per day in the studied period in absence of mandate. The post-mandate announcement trend in first doses turns less steep in the three countries (the estimate is positive, although only statistically significant for Germany), which suggests a lack of net intertemporal substitution as of October 31. On the other hand, the estimates are statistically significantly negative for Alberta, New Brunswick, Saskatchewan and Prince Edward Island, which is consistent with the mandate effect on new first doses diminishing over time in these provinces.
In Supplementary Table 9, we report additional results from two alternative time-series specifications: using weekly deaths and average hospitalizations as information control variables or using a binary, instead of weekly-averaged, policy variable. The results are very similar to those in Table 2 and confirm the large and positive estimated effect of mandate announcements on first dose vaccine uptake. We provide further details on the time-series model fit and out-of-sample projections in Supplementary Information.
Counterfactuals
We use our estimates of the mandate effect on vaccine uptake from Table 2 to compute the cumulative increase in new vaccinations for each province and for France, Italy and Germany, relative to the counterfactual scenario of absence of mandate. Counterfactual weekly and cumulative doses are computed as explained in Methods. We report the vaccinations gains both in levels (mln doses) and in percentage points relative to the vaccination rate at the mandate announcement date.
We find large cumulative increases in the first-dose vaccination rate for all provinces relative to in the absence of mandate: from 1.9 p.p. (90% CI -0.3– 3.0) in Ontario to 5 p.p. (90% CI 4.0–5.6) in Saskatchewan and 5 p.p. (90% CI 3.9–5.8) in New Brunswick, with all other provinces in between, as of October 31, 2021; see Fig. 4 and Fig. 5. These estimated total gains in uptake add up to 2.9 p.p. (90% CI 1.3–3.8) of the eligible population or 979,000 new first doses (90% CI 425,000–1,266,000) for Canada as a whole; see Supplementary Table 10. This is a significant increase in vaccine uptake considering the relatively short period in which it was achieved (within 6 to 10 weeks of the mandate announcement for most provinces; 13 weeks for Quebec) and the very high pre-mandate first-dose vaccination rate in Canada (over 80% on average at the time of the mandate announcements, see Supplementary Table 2). For example, Saskatchewan’s 5 p.p. estimated gain amounts to about one quarter of all remaining unvaccinated eligible people in the province as of the mandate announcement date (Sep. 16, 2021).
We do not find evidence of net intertemporal substitution (decrease in the cumulative first-dose gains from pulling vaccinations forward in time) as of October 31, 2021, except in PEI. However, the strongly negative estimates in Table 2 and the trends in Extended Data Fig. 9 indicate that Alberta, New Brunswick and Saskatchewan are projected to exhibit net intertemporal substitution after the end of October 2021. That said, given the flattening of the counterfactual curves on Fig. 4, the cumulative gains are likely to remain significant.
In Extended Data Fig. 10, we also plot a counterfactual computed using the DID estimate for Canada from Table 1 (i.e., using the same policy estimate for all provinces). We find that, as of the DID sample end date of September 14, 2021, the five Canadian provinces that announced mandates by that date have benefited from 287,000 additional first doses (90% CI 239,000–333,000) or a vaccination rate increase of 0.9 p.p. (90% CI 0.7–1.0) for the eligible population, relative to the no-mandate counterfactual.
We also estimate large increases in first-dose vaccinations relative to the no-mandate counterfactual in the three countries: 8 p.p. (90% CI 4.3–10.8) or 4.59 mln (90% CI 2.47–6.25) additional first doses in France, 12 p.p. (90% CI 5.0–15.1) or 6.48 mln (90% CI 2.67–8.14) doses in Italy, and 4.7 p.p. (90% CI 4.1–5.1) or 3.47 mln (90% CI 3.06–3.81) doses in Germany as of October 31, 2021 using our main Table 2 specification, see Fig. 5, Fig. 6 and Supplementary Table 10. These estimated gains are larger than those for Canada, possibly because of the earlier mandates in these countries or the lower starting vaccination rates, and they may partly reflect expanding the scope of the initial mandates, e.g., to inter-regional travel and employment in September and October in Italy.
The counterfactuals assume that all explanatory variables except the mandate announcement (e.g., cases, deaths or time trends) remain fixed at their observed values and that the model parameters remain stable. These assumptions are more plausible over relatively short periods. Hence, these simulations should be interpreted primarily as an illustration of the estimated impact of the mandates on vaccine uptake rather than policy guidance.
Discussion
We find that government-mandated proof of vaccination requirements or certificates have sizable and statistically significant impact on COVID-19 vaccine uptake, with large observed increase in first-dose vaccinations in the first several weeks after mandate announcement and lasting cumulative gains relative to the pre-announcement trend. This includes robust difference-in-differences evidence using the variation in the timing of mandate announcements within the same country, Canada. Our results are of similar magnitude and consistent with the findings of other authors, using different data samples and methodologies: [26] use a synthetic control approach and estimate large increases in vaccinations in France (8.6 mln), Italy (4 mln) and Israel (2.1 mln) from 20 days prior to 40 days after COVID-19 certification mandate implementation, while [27] compute counterfactuals based on an innovation diffusion model of vaccine uptake and attribute vaccination rate increases of 13 p.p. for France, 6.2 p.p. for Germany and 9.7 p.p. for Italy to the announcement of COVID-19 certificate requirements, with associated additional gains from averted deaths and GDP losses.
The estimated mandate effect on uptake varies across the Canadian provinces, with the timing of announcement, implementation and the percentage unvaccinated playing a role. The estimated impact also differs across France, Italy and Germany. Further research on understanding this heterogeneity and on other potentially important factors (e.g., the role of government communication or the media, the degree of political polarization, the amount of social trust) can complement our study.
The unambiguous and large increases in vaccine uptake we find compares favourably to the mixed evidence from using financial incentives (cash, gift cards, lotteries), [12–17], or behavioural nudges, [12, 18–20]. Financial incentives for vaccination have been criticized for the optics of putting a low dollar value on being vaccinated compared to the social benefits, because of the perceived unfairness in rewarding people who delayed their vaccination, or because of potential moral hazard problems (e.g., expecting future payments). Financial incentives may even have the perverse effect of validating vaccine concerns among unvaccinated individuals; see [12]. Others [28] have argued that, given the already high vaccination rates in developed countries, behavioural nudges may not be very effective, which is consistent with the findings of [12] and [13]. On the other hand, vaccination mandates have been controversial, as some people perceive them as restrictions on personal freedom. This can affect compliance and increase both the direct implementation and enforcement costs, as well as the political costs of introducing a mandate.
In terms of external validity, in 2020, [29] conducted a survey on vaccine hesitancy across 19 countries comprising around 55% of the world population. The participants were asked: “If a COVID-19 vaccine is proven safe and effective and is available, will you take it?” Canada, Italy and Germany placed around the middle in the self-reported vaccine hesitancy rate (29% to 31%), while France had a higher hesitancy rate (41%). In this regard, our results on the large impact on new first dose vaccinations in these countries can be useful to public health authorities in other places looking for an effective strategy to increase vaccine uptake.
We conclude by listing some limitations and areas for further research. We have abstracted from assessing the mandates’ impact on health outcomes (cases, hospitalizations or deaths). However, vaccine effectiveness estimates from the medical literature can be used to study this further, ideally controlling for possible changes in behaviour. Our focus is on vaccination and certification requirements for non-essential settings although some locations, e.g., Italy, expanded the mandates to inter-regional travel or employment and other countries (Greece, Austria) have proposed even broader mandates, which we do not analyze. For data limitation reasons, we also could not study the effect by age group as in [26] or the possible role of past personal sickness outcomes as in [30]. See also Methods for limitations and required assumptions related to the time-series analysis and counterfactuals and our ways of tackling them.
Naturally, a full cost-benefit analysis of proof of vaccination mandates is beyond the scope of this paper. In particular, the costs of imposing and enforcing the mandates – economic, political, or personal – are very hard to estimate, as is the social value of vaccinating an additional person, [31, 32]. One component of the latter is avoided healthcare costs, e.g., [14] estimate that the Ohio vaccine lottery saved the state USD 66 mln in ICU costs. Our results are a step toward quantifying the benefits of proof of vaccination requirements.
Methods
Data and definitions
We use data on COVID-19 vaccination numbers, cases and deaths for all ten Canadian provinces and for France, Italy, Germany and Spain; see Supplementary Table 11 for details. We collected the Canadian data from the official provincial dashboards or equivalent sources. We use the Our World In Data (OWID) dataset for country data. Announcement and implementation dates of the proof of vaccination mandates or certificates were collected from government websites and major newspapers; see Supplementary Table 1.
The main variables used in our statistical analysis are defined next. Every-where, i denotes province or country and t denotes time measured in days (date). We aggregate the data on vaccinations, cases and deaths on a weekly basis (totals for the week ending on date t, i.e., dates t − 6 to t) to reduce the influence of day-of-the-week effects or reporting artifacts (e.g., lumping week-end data in Monday’s report; in the latter case, we distribute the reported total equally over the affected dates).
Outcome, Vit. The main outcome variable is the logarithm of administered vaccine first doses per 100,000 people, for the week ending at t (dates t − 6 to t). We use first doses, as they most directly reflect the impact of the mandates on the intent to be immunized and avoid potential issues related to second dose scheduling or availability. In Supplementary Table 7, we also report results using second doses as the outcome. All COVID-19 vaccines used in Canada during the study period, namely Pfizer (Comirnaty), Moderna (SpikeVax) and AstraZeneca (Vaxzevria), were originally considered two-dose. Using the logarithm of weekly first doses allows us to interpret the regression coefficients as percentage changes; moreover, the estimates are invariant to normalization, e.g., by population (subsumed in the regression constant or fixed effects). We thus use “log weekly first doses” for simplicity throughout the text, except where the actual scale is important.
Policy, Pit. Let be the mandate announcement date in jurisdiction i. We construct a binary policy variable Pit equal to 1 for all post-announcement dates t ≥ and equal to 0 for all . Proof of vaccination mandates were announced in all ten Canadian provinces over the period Aug. 5, 2021 to Sep. 21, 2021; see Extended Data Fig. 4 and Supplementary Tables 1 and 12. Last to announce were the four Atlantic provinces, which had the lowest per capita case rates in August 2021, and Alberta and Saskatchewan, which had the highest per capita case rates in August 2021; see Extended Data Fig. 3. France, Italy and Germany also introduced proof of vaccination certificates in July – August 2021; see Supplementary Table 1.
Information, Iit. We include control variables related to the concurrent COVID-19 epidemiological situation, specifically log of weekly cases, Cit, and log of weekly deaths, Dit, for the week ending at date t (dates t − 6 to t). We refer to these variables jointly as ‘information’, Iit (see [33, 34]) since they can inform a person’s COVID-19 exposure risk assessment and/or decision to be vaccinated, e.g., as shown in [21, 22]. Another possible information variable is hospitalizations. However, it is strongly correlated with COVID-19 cases and deaths so all three cannot be included at the same time (see Supplementary Tables 5 and 9 for results using deaths and hospitalizations as information). To address zero weekly values, which sometimes occur in the smaller provinces for deaths or cases (4.4% of all observations for cases and 10.7% for deaths), we replace log(0) with −1, as in [33].
Other controls, Wit. We include province fixed effects and date fixed effects in our difference-in-differences analysis. The province fixed effects account for any time-invariant province characteristics such as sentiment towards vaccination, age structure, education, political alignment, etc. The date fixed effects control for national trends or events, e.g., public messaging, vaccine-related international travel regulations, or campaigning for the 2021 federal election. In the time-series analysis, we control for time trends.
Time period. We use the period June 15, 2021 to October 31, 2021. The start date was chosen to ensure that possible constraints in obtaining a first dose related to eligibility or supply are minimal or non-existent. This helps avoid potential bias from constrained vaccine supply affecting the pre-or post-mandate pace of vaccination. In Canada, the provinces opened registration for first-dose vaccination for any person of age 12 or older between May 10, 2021 in Alberta and May 27, 2021 in Nova Scotia; see Supplementary Table 3. First-dose availability in France, Italy and Germany was similar by mid-June, at least for the 18-plus age group. We explore different sample start dates in robustness checks; see Extended Data Fig. 5. The sample end date is based on data availability at the time of statistical analysis and writing. First doses for the 5–11 age group were not approved in the studied period.
Difference-in-differences estimation
We estimate a behavioural model in which the decision to receive a COVID-19 vaccine, measured by new weekly first doses, is affected by the policy setting, Pit (whether a mandate has been announced) and current COVID-19 epidemiological information about public health conditions, Iit, proxied by weekly cases and deaths. Based on the raw data patterns in Fig. 1 and Extended Data Fig. 3 and the absence of vaccine supply constraints in the studied period, we assume no lag between a mandate announcement and a person’s ability to receive a first vaccine dose and no information lag. We perform sensitivity analysis using alternative lags (see Extended Data Fig. 5); the results affirm our baseline choice of no lag.
In the difference-in-differences (DID) analysis, we estimate: where Vit is log weekly first doses for location i and date t, Iit is information, γt are date fixed effects, δi are province fixed effects, and εit is an error term. The coefficient α on the policy variable Pit captures the average effect of the mandate announcement on weekly first doses over all post-announcement dates. To capture dynamic effects of the mandates, we also estimate a version of equation (1) with Pit split into separate indicator variables, one for each week after the mandate announcement date; see Table 1.
To correct for the small number of clusters in the estimation since there are only ten provinces, we report “wild bootstrap” p-values (we use the Stata package boottest clustered by province with 4,999 repetitions, see [35, 36]). The use of clustered standard errors allows the error terms to be to be serially correlated within each province. Alternative methods for computing the standard errors are explored in Supplementary Table 6 – clustering at the province level (Stata command “cluster”), wild bootstrap standard errors clustered at the province level and wild bootstrap standard errors two-way clustered by province and date allowing for spatio-temporal correlation (Stata command “boottest”).
The recent methodological literature, [23, 37–40], has argued that the standard OLS two-way fixed effects (TWFE) estimator can be invalid in panel-data settings with staggered adoption like ours if there is heterogeneity in the treatment effect across cohorts (provinces in our data) and/or over time. The reason is that the TWFE estimate is a weighted average of many 2-by-2 DID treatment effects, where some of the weights can be negative or incorrect because of contamination from other periods.
In particular, Sun and Abraham [23] develop a difference-in-differences estimator valid under these conditions, which we use to estimate (1). The Sun and Abraham estimator uses never-treated or last-treated units as the control group and is constructed as weighted average of treatment effects for each cohort (by date of mandate announcement) and each relative time after or before the announcement (we use the Stata function eventstudyinteract provided by the authors). Specifically, to calculate the average treatment effect α, we replace αPit in (1) with where C is the set of never-treated or last-treated provinces (control group), is a “relative time” indicator, and is the date of treatment for province i. Under parallel trends and no anticipation, [23] show that αi,l is consistent for the province-time specific treatment effect. Then, the average treatment effect for each relative-time period, αl, is the appropriately weighted (by the sample share of each treated province in relative time l) average of αi,l across the units i, and α in (1) is computed as the simple average of αl across l.
The estimator requires excluding all time periods in which units in the control group are treated. Since the last province to announce a mandate was Prince Edward Island, on September 21, 2021, we can use only data until September 20, at the latest, see Supplementary Table 1. We present results for different control groups and corresponding sample periods in Supplementary Table 4. We also compare our main results with the OLS TWFE estimates in Supplementary Table 5.
Time-series estimation
We estimate the following ‘interrupted time-series analysis’, [24], specification for each country or province (we omit the subscripts i for simplicity since all variables refer to the same location): where Vt are log weekly first doses for the week ending at date t, c is a constant, It is proxy for information, analogous to its counterpart in equation (1), and ηt is the error term. In our baseline specification in Table 2, we construct the policy variable Pt as the weekly average from date t − 6 to t of the ‘mandate announced’ indicator Pit defined above. This is consistent with the weekly vaccination and information variables Vt and It and improves the fit in the time-series regressions. We also present results without weekly averaging in Supplementary Tables 9 and 10. We include 7-day and 14-day lagged values of Vt (instead of first and second lags) since the outcome variable Vt is a weekly total.
We include two time trends in equation (2): Tt is a linear daily time trend initialized at the sample start date t = 0 and is an ‘interaction’ time trend which takes value 0 at all dates until the announcement date (inclusive) and increases by 1 for each day afterwards. The coefficients τ1 (slope) and τ2 (change in slope at the announcement date) characterize respectively the baseline (pre-announcement) time trend (with slope τ1) and the post-announcement time trend (with slope τ1 + τ2) in weekly first doses.
Standard errors and p-values are calculated using the Newey-West [41] heteroskedasticity and autocorrelation (HAC) robust estimator, with 3 lags. The lag was chosen as the closest integer to T 1/4, where T is the sample size.
The time-series approach requires stronger identification assumptions than the DID approach since there is no control group. Specifically, we need to assume exogeneity of the announcement date and that the time-series process for weekly first doses changes after the announcement only because of the policy, i.e., it would have followed the same pre-trend if no mandate had been announced. We also cannot control for fixed effects in a flexible way beyond including a constant and time trends. Under these assumptions, the coefficient on the policy variable Pt captures the average shift in first-dose uptake attributed to the mandate announcement. Similarly, the coefficient on the interaction time trend measures the slope change in the trend of first doses after the mandate announcement relative to the pre-announcement trend.
Structural break at the announcement date
We perform a structural break Chow test, [42], for a known break point using the log weekly first doses, Vt in Supplementary Table 8. The presence of a break point at the mandate announcement date indicates an abrupt change or shift in the first-dose time series process. We use a bandwidth of 50 days before and 35 days after the announcement date. The unequal before-after bandwidth is chosen to reduce the size distortion of the test, since the outcome variable is a weekly sum and the error terms are serially correlated; see [43]. Columns (1) and (2) in Supplementary Table 8 use the log of weekly first doses, Vt while column (3) uses first-differenced weekly first doses. The differenced series is stationary, and the error terms are not serially correlated, which alleviates concerns about size distortion in the test. The power of the Chow test is weaker in this specification since the first-differenced Vt series is a growth rate being used to test for a level shift. The differenced series is also noisier as it captures daily fluctuations. Overall, the structural break test results show that a mandate announcement is strongly associated with a trend break in first-dose vaccine uptake in all or most locations.
Counterfactuals
Calling the mandate announcement date, we compute the counterfactual log weekly doses per 100,000 people, iteratively using where is the coefficient estimate of the mandate announcement variable Pt from column (1) of Table 2, is the estimate of the interaction time trend from column (2) of Table 2, and where the terms multiplied by the estimates and account for the the lagged values Vt−7 and Vt−14 in equation (2). The policy effect estimate , as well as and are specific for each respective country or province. To plot Extended Data Fig. 10, we compute the counterfactual mean as where is the DID estimate on the policy variable Pit from Table 1, column (2).
Data availability
All data used in this paper are publicly available at https://github.com/C19-SFU-Econ/dataV
Data Availability
All data used in this manuscript is based on information available in public domain. Data sources and links are provided within the manuscript.
Author contributions
All authors contributed equally to this research.
Competing interests
All authors declare no conflict or competing interests arising from this research. The authors have not received funding for this research.
Extended Data
Supplementary information
Robustness analysis details
Alternative initial dates
In Fig. 5, upper right panel, we show that our estimates of the effect of a mandate announcement on first dose vaccine uptake are not very sensitive to the choice of initial sample date between May 1 and Jul. 15, 2021. This provides further reassurance that vaccine supply constraints are not a major concern for our results.
Lags
In (1), we assume no lag between the mandate announcement Pit and a person’s ability to receive first dose (the outcome Vit). In practice, a small delay may occur (e.g., from booking an appointment to receiving the vaccine) even in absence of supply constraints. The lower left panel of Fig. 5 displays the estimates of the mandate effect when assuming zero lag (our baseline) to 7-day lag. The policy estimates remain large and statistically significant when varying the lag length, with a slight decrease for longer lags. The lower right panel of Fig. 5 displays the policy estimates when assuming a lag of zero (our baseline) to 14 days in the information variables Iit (log weekly cases and deaths). The mandate effect estimates remains large and statistically significant when varying the lag length. We also checked a specification in which we lag the information variables Iit (log weekly cases and deaths) with the same lag as the policy Pit, and the results are nearly identical.
Other robustness checks
We perform additional robustness checks in Table 5. For easier comparison, column (1) replicates the baseline estimates from columns (2) and (3) of Table 1. Column (2) of Table 5 omits cases and deaths, the coefficients for which are not significant in the baseline specification. In column (3), we control for log weekly deaths and log average weekly hospitalizations as information. Column (4) reports the estimates of a weighted specification using the provincial populations as weights and the baseline information variables (cases and deaths). The estimates are smaller than in the unweighted baseline (column 1), suggesting that smaller provinces contribute more to the the estimated effect of mandate announcements on vaccine uptake. In column (5), we re-estimate equation (1) using the standard OLS two-way fixed effects (TWFE) estimator which, as discussed in Methods, can be biased in the presence of heterogeneous treatment effects over cohorts and/or time. In our setting, we find that these estimates differ relatively little from our baseline Sun-Abraham treatment effect heterogeneity robust estimates.
In Table 5 column (6), we use log of daily (instead of weekly) first doses as the outcome and log of daily cases and deaths as information and find similar estimates to those in our baseline specification, although larger in week 0 since the outcome is not a moving sum. In column (7), we use the level of weekly first doses per 100,000 people as the outcome. The estimates are noisier than in the baseline and indicate that a mandate announcement increases weekly first dose uptake by roughly 482 per 100,000 people on average (CI -11–975, p < 0.06) after the announcement. Overall, all these alternative specifications confirm the robustness of our main finding of statistically significant and large impact of mandate announcements on vaccine uptake.
We also ran a specification replacing the province fixed effects with time-invariant province characteristics (GDP per capita, % rural population, population density, unemployment rate, ICU beds per capita, % of population of age 65 or older, full-time university students per capita, and political alignment). The estimated effect of the mandate announcements on vaccine uptake in this specification is very similar in magnitude and significance to our main estimates in Table 1.
Randomization inference
Difference-in-differences inference from the model in equation (1) relies on asymptotic approximations requiring a sufficiently large number of provinces. While we do account for the fact that there are only 10 provinces in our sample by using wild bootstrap to compute the estimates’ p-values, as an additional robustness check, we implement a variant of Fisher’s randomization test, [44]. Specifically, we estimate equation (1) 5,000 times by randomly assigning the mandate announcement date for each of the ‘treated’ provinces in our baseline specification. Fig. 6 plots the histogram of the randomized inference (placebo) mandate effect estimates, along with the 95% confidence band. Only 2.88% of the 5,000 placebo estimates are larger than our baseline estimate 0.506 from column (2) of Table 1 (the solid vertical line on Fig. 6), offering assurance that our main result does not hinge on asymptotic inference.
Mandate implementation
Our main results use the regional variation in the timing of announcement of proof of vaccination mandates to examine their effect on vaccine uptake. In addition, there is variation in the interval between the mandates’ announcement and implementation (coming into force), as well as in the implementation dates themselves – see Table 1 and Fig. 3. We checked whether we can detect additional effects on first-dose uptake, associated with the mandates’ implementation, by adding to equation (1) an indicator variable that equals to 1 post-implementation and 0 otherwise. The mandates were implemented in all provinces between Sep. 3 and Oct. 22, 2021. The estimate on this indicator variable is positive but not statistically significantly different from zero (not displayed). However, we cannot conclude that there do not exist implementation effects, since even the latest feasible sample end date (Sep. 20, 2021) necessitated by the requirement to have a not yet treated control group is only soon after the mandate implementation date in 3 of the 10 provinces (see Table 1).
Potential spillover effects
To check for potential spillover effects whereby a mandate in one province may have an effect on vaccine uptake in another province, we ran the time-series regression, equation (2), for Ontario and British Columbia (BC) assuming earlier announcement dates from other provinces. Specifically, the regression for Ontario using BC’s announcement date (Aug. 23) instead of Ontario’s actual announcement date (Sep. 1) yields a statistically insignificantly different from zero policy estimate, . Similarly, regressions for BC and Ontario using the Quebec announcement date (Aug. 5) yield 0.182 (p = 0.263) and -0.451 (p = 0.019) respectively. While this evidence does not support the presence of spillover effects, if they did exist across some provinces or dates, then we can interpret our estimates as a lower bound of the full policy effect, as some fraction of people would have been vaccinated already before the announcements.
Time-series fit and out-of-sample projection
In Supplementary Fig. 1, we display the time-series model fit with the data until Oct. 31, 2021. In addition, we plot out-of-sample predictions obtained using equation (2), the estimates from Table 2 and the actual weekly cases and deaths data for these out-of-sample dates. The time-series specification fits the weekly first dose data very well (e.g., the adjusted R-squared is 0.997 for France, 0.940 for Italy and 0.990 for Germany), and the out-of-sample projections (dotted line) remain close to the observed first-dose data from November, with the exception of Germany, where the COVID-19 certificate became more restrictive in this period by excluding unvaccinated people that were previously able to supply a negative test from indoor dining, bars, hairdressers, etc. A constraining factor for the out-of-sample projections is that first doses for the 5-to-11 age group were approved in November (Nov. 19 in Canada and Nov. 25 in the EU), and, for many jurisdictions in our sample, we cannot distinguish these doses from adult first doses in the vaccinations data. Hence, we only show out-of-sample projections for three additional weeks beyond our end-sample date, until Nov. 21, 2021.
Time-series – placebo announcement dates
Similar to the DID randomization inference robustness exercise (Fig. 6), we use our time-series model to estimate the mandate policy effect (the parameter π in equation (2)) for a range of placebo announcement dates (132 unique dates in total including the true announcement date), from Jun. 17 to Oct. 24 for each of France, Italy and Germany. Supplementary Fig. 2 displays the results. There is only a small number of placebo dates (mostly in a close neighbourhood of the true announcement date) for which the time-series regression yields a larger policy estimate. Specifically, there are 12 dates (9.1%) with estimate larger than that for the true announcement date for France, 14 dates (10.6%) for Italy and only 5 such dates (3.8%) for Germany. While these results are reassuring (although relatively noisy for France and Italy), they should only be treated as supplementary, since the time-series approach and specification we use rely on a known date for the policy regime change.
Related literature – details
[26] use a synthetic control approach to evaluate the impact of COVID-19 certification mandates on vaccine uptake, overall and by age group, in six countries. The authors estimate significant increases in vaccinations in France (about 8.6 mln), Italy (4 mln) and Israel (2.1 mln) from 20 days prior to 40 days after the mandate implementation. While the main takeaway is similar, our paper differs in several important ways. First, we provide robust difference-in-differences evidence using the variation in mandate timing within the same country, Canada, in addition to evidence using time-series methods for France, Italy and Germany. Second, instead of total vaccinations, we focus on first doses, as most directly reflecting the decision to become immunized and avoiding potential issues related to second dose scheduling or availability in spring/summer 2021. This is one possible reason why, unlike [26], we find a statistically significant increase in vaccine uptake in Germany around the mandate. When using total vaccinations instead of first doses, we also do not find a statistically significant effect for Germany. Third, instead of using a fixed 20-day cutoff before implementation, we use the actual mandate announcement date (the interval between mandate announcement and implementation in the ten Canadian provinces and three countries in our sample varies from 5 to 41 days) as the intervention indicator and show a strong impact on vaccine uptake thereafter.
There is little evidence that financial or behavioural nudges increase the vaccination rate among hesitant people. [12] report results from a mid-2021 randomized controlled trial (RCT) with unvaccinated members of a large Medicare health plan in a California county with 77% vaccination rate at the time of the study. The authors examine the effect of $10 or $50 financial incentives, different public health messages and an appointment scheduler, on vaccination intentions and uptake within 30 days of the intervention. They find that none of the financial or behavioural treatments increased the vaccination rate among the treated. The proof-of-vaccination mandates we analyze target a similar group of unvaccinated people that have had the opportunity to be vaccinated for a long time. The population-weighted average first-dose vaccination rate on the dates of the provincial mandate announcements is 83.3%, even higher than in [12], suggesting that financial or behavioural incentives are even less likely to be effective in our setting. In an RCT study in Sweden, [13] find that a modest payment of SEK 200 (USD 24) is associated with a 4.2 p.p. increase in vaccinations, relative to a baseline rate of 71.6% in the control group, while none of three behavioural nudges had impact. However, the RCT subjects were yet to become eligible to be vaccinated, and hence the setting is not comparable to ours in which weeks have elapsed after widespread vaccine availability.
Footnotes
Minor extensions and change in presentation from a social science style to a natural science style.