Abstract
Background In Canada, all provinces implemented vaccine passports in 2021 to increase vaccine uptake and reduce transmission in non-essential indoor spaces. We evaluate the impact of vaccine passport policies on first-dose COVID-19 vaccination coverage by age, area-level income and proportion racialized.
Methods We performed interrupted time-series analyses using vaccine registry data linked to census information in Québec and Ontario (20.5 million people ≥12 years; unit of analysis: dissemination area). We fit negative binomial regressions to weekly first-dose vaccination, using a natural spline to capture pre-announcement trends, adjusting for baseline vaccination coverage (start: July 3rd; end: October 23rd Québec, November 13th Ontario). We obtain counterfactual vaccination rates and coverage, and estimated vaccine passports’ impact on vaccination coverage (absolute) and new vaccinations (relative).
Results In both provinces, pre-announcement first-dose vaccination coverage was 82% (≥12 years). The announcement resulted in estimated increases in vaccination coverage of 0.9 percentage points (p.p.;95%CI:0.4-1.2) in Québec and 0.7 p.p. (95%CI:0.5-0.8) in Ontario. In relative terms, these increases correspond to 23% (95%CI:10-36%) and 19% (95%CI:15-22%) more vaccinations. The impact was larger among people aged 12-39 (1-2 p.p.). There was little variability in the absolute impact by area-level income or proportion racialized in either province.
Conclusions In the context of high baseline vaccine coverage across two provinces, the announcement of vaccine passports led to a small impact on first-dose coverage, with little impact on reducing economic and racial inequities in vaccine coverage. Findings suggest the need for other policies to further increase vaccination coverage among lower-income and more racialized neighbourhoods and communities.
Key messages
Vaccine passport policies increased COVID-19 vaccination coverage by approximately 1 percentage point (19 to 23% increase in vaccinations) in Québec and Ontario, Canada.
Although vaccine passport policies increased vaccination coverage, absolute gains were limited in the context of high prior vaccine coverage.
Vaccine passports had little impact on reducing economic and racial inequities in vaccine coverage.
Introduction
The threat posed by the coronavirus disease (COVID-19) pandemic led to unprecedented interventions, including both generalized and targeted restrictions. Once vaccines became more widely available in 2021 in high-income countries, many governments implemented proof-of-vaccination policies to further mitigate the pandemic’s impacts on population health and the economy.1 Often termed “vaccine passports”, these policies generally required demonstration of vaccination status or a valid exemption to access non-essential activities and spaces, including restaurants, bars, movie theatres, and concerts.
In Canada, all provinces and the Yukon territory introduced vaccine passports in 2021 and discontinued the policy in the first half of 2022. Québec and Ontario —the two most populous provinces with a population of 22.7 million and Canadian epicenters of the pandemic—2–4 were among the first to announce vaccine passports. Provincial governments stated that these policies aimed to reduce SARS-CoV-2 transmission and prevent re-closure of non-essential venues by increasing vaccination coverage and limiting contacts of individuals who had not yet been vaccinated in non-essential venues.5–7
The ethical and practical implications of vaccine passports have been debated,1,8–11 yet evidence on their effectiveness at incentivizing and increasing coverage of COVID-19 vaccination remains limited. Studies in Europe and Canada found that the introduction of vaccine passports led to increases in vaccination uptake, but this impact depended on age and prior vaccine coverage.12–14 These studies have been limited by their use of provincial- or national-level data that restricted exploration of heterogeneity by age and —more importantly— have not examined the effects of vaccine passports according to social determinants of health (SDOH). Given that the COVID-19 pandemic has disproportionately affected communities experiencing social and economic marginalization,4,15 it is essential to examine whether vaccination policies resulted in socioeconomic disparities in coverage.
Using vaccine registry data linked to area-level census information, we evaluated the impact of vaccine passports on first-dose vaccination coverage in Québec and Ontario using an interrupted time-series methodology. For each province, we estimated the impact of the vaccine passport by age, and two area-level social determinants: income and proportion racialized.
Methods
Study setting and population
In both Québec and Ontario, following early prioritization of residents of long-term care homes and health care workers, vaccination of the general adult population (≥18 years) and youth aged 12-17 years began in May 2021 with BNT162b2 (Pfizer-BioNTech), mRNA-1273 (Moderna), or ChAdOx1 (Oxford-AstraZeneca).16–19 COVID-19 proof-of-vaccination policies (herein “vaccine passports”) were announced on August 5th (Québec) and September 1st (Ontario) and came into full force on September 15th (Québec) and September 22nd, 2021 (Ontario).6,7,20 Non-essential activities and venues targeted by these policies were similar in both provinces (e.g., restaurants, bars, concerts, gyms, school extracurriculars), and restrictions applied to those aged 13 (Québec) or 12 (Ontario) years and over.
Data sources and measures
We obtained vaccination data from the Registre de vaccination du Québec and Ontario’s COVax system,21,22 which include individuals’ dose administration date, age, and address or dissemination area (DA) of residency. Data were aggregated at the DA level — the smallest standard geographic area for which census information is available (average 400-700 residents).23 We included all individuals aged 12 years and over (population eligible for vaccination at time of announcement). Age was categorized based on vaccination priority (12-17, 18-29, 30-39, 40-49, 50-59, and 60+ years). We computed the weekly vaccination rate by DA and age group, defined as the number of first doses administered per 100,000 people without a first dose. We evaluated first-dose coverage because it may better capture people’s response to vaccination mandates, whereas second-dose coverage depended primarily on the time since first dose. Additionally, second-dose eligibility would be affected by changes in the recommended dosage interval (initially 16 weeks but shortened during the summer of 2021).24–27
We obtained DA-level after-tax income, per person equivalent from the Postal Code Conversion File Plus Version 7A/7D,28 and the proportion racialized (based on self-reported “visible minority”) from the latest available Canadian Census (2016) at the time of analysis.29 Income was ranked at the census metropolitan area level (to account for within-province variability in cost-of-living) from lowest (1) to highest (5), while proportion racialized was ranked at the provincial level from highest (1) to lowest (5). This ordering was chosen such that the first quintile would align with observed data on the highest incidence of COVID-19 cases.4,15 The ranking balanced the population in each quintile (i.e., each quintile had approximately one fifth of the total population).
Study design and statistical analysis
Analyses were stratified by province. We performed interrupted time-series (ITS) analysis to estimate the impact of the vaccine passport by modeling a counterfactual scenario based on the pre-intervention temporal trend.30,31 We allowed for changes in both level and slope of the vaccination rate as a result of the announcement of the vaccine passports. The change lasted for six weeks in both provinces, starting on August 14th (Québec) or September 4th (Ontario). Québec’s date was lagged by one time unit because inspection of the raw data suggested that changes in the weekly rate following the announcement were only detectable after one week, likely because of the announcement timing and decreased vaccination during weekends. The study period was from July 3rd (chosen to align with the end of school year) to five weeks after the end of the vaccine passport’s impact period (i.e., October 23rd for Québec and November 13th for Ontario).
Our modeling approach consisted of two steps. First, we used negative binomial regressions with a natural spline to capture the pre-announcement trend of the DA-level weekly vaccination rates, adjusting for baseline vaccination coverage (i.e., July 3rd 2021; categorical with six groups).32,33 Second, we used estimated model coefficients to obtain counterfactual vaccination rates and coverage in the absence of the vaccine passport. We computed the absolute impact of the vaccine passports (observed minus counterfactual vaccination coverage) at the end of the study periods (Québec: October 23rd, 2021; Ontario: November 13th, 2021). In addition, we calculated the relative increase in vaccine doses administered between the passport announcement and the end of the study period.
We investigated heterogeneity in the impact of vaccine passports by age12 and by area-level SDOH, which have been associated with higher COVID-19 infection burden.4,15 We fit three models in which the vaccine passport impact could vary by age group, area-level income, or area-level proportion racialized. To further examine trends by SDOH, we fit two additional models with interaction terms between age group and area-level income or proportion racialized. We evaluated heterogeneity of impact by assessing trends in absolute and relative impacts of the vaccine passport by age and SDOH. To examine the impact of vaccine passports on inequities in vaccination coverage, we focused on the magnitude of the absolute impact.
Since heterogeneities in the impact of vaccine passports could be influenced by differences in baseline vaccination coverage, we re-fit the first three models with an interaction term between baseline coverage and the impact of the vaccine passports. We then re-estimated the absolute impact of vaccine passports while holding baseline coverage constant (i.e., setting the baseline variable for all DAs to the same value).
Finally, Montréal and Toronto —the largest census metropolitan areas in each province— were important epicenters of SARS-CoV-2 transmission. Their sociodemographic profiles also differed from the rest of their provinces, which could have affected the impact of vaccine passports. Therefore, main analyses were replicated restricting to the DAs encompassed by these two census metropolitan areas (with the DAs re-ranked according to SDOH).
Confidence intervals (CIs) were obtained using 1,000 bootstrap replicates, using census tracts as the resampling unit to account for geographical and temporal correlations. The 95% CIs were computed by taking the 2.5th and 97.5th quantiles.
Sensitivity analyses
We explored how alternative modeling choices affected model fit and results by re-parametrizing the age model.34 Briefly, we assessed if the chosen start of the study period influenced conclusions by changing the start of the time-series (±1 week), assessed the robustness of our results to different lengths of the vaccine passport impact period (5 or 7 weeks), and examined different model specifications for the time trend (using non-spline methods). Fits were compared based on Akaike Information Criterion, Bayes Information Criterion, and visual assessment.
All analyses were carried out in R V.4.1.0, using packages fixest and splines.35–37 Full details on our modeling approach, model equations, and sensitivity analyses can be found in Supplementary Materials.
Results
Observed COVID-19 first-dose vaccination coverage over time
In both provinces, first-dose COVID-19 vaccination coverage was 82% in the eligible population (≥12 years) when the vaccine passport was announced. Coverage was highest among people aged 60+ years, with 94% and 87% coverage in Québec and Ontario, respectively, while coverage for those aged 12-17 years was 68% and 76%, respectively. By the end of the study period, vaccination coverage had increased by about 5 percentage points (p.p.) in each province (Table 1).
Pre-announcement vaccination coverage in the lowest-income DAs was 9 and 7 percentage points (p.p.) lower than highest-income DAs in Québec and Ontario, respectively (similar inequalities in Montréal and Toronto). There were also disparities by proportion racialized: vaccination coverage in DAs with the highest-proportion racialized was 4 p.p. and 8 p.p. lower than the lowest-proportion ones in Québec and Montréal, respectively. In Ontario, these inequalities were reversed — vaccine coverage was 3 p.p. higher in the highest-proportion racialized as compared to the lowest ones, and there was little difference in Toronto (<1p.p.; Supplementary Table S2).
Observed pre- and post-passport announcement vaccination rates
Prior to the announcement of the vaccine passports, weekly first-dose vaccination rates were stable in Québec and declining in all age groups in Ontario (Figure 1). Increased vaccination rates were observed in both provinces in the week that followed the announcement of the passports, especially among younger age groups (12-17 and 18-29 years old). Comparable increases occurred across income and proportion racialized quintiles. These increases were sustained over a period of six weeks. Similar patterns were observed for Montréal and Toronto (Supplementary Figure S1).
Observed (points) and modeled (blue and yellow) vaccination rates over time are shown. Predicted vaccination rates were obtained from three different regression models where the vaccination rate and the impact of the vaccine passport were allowed to vary by age group (A, D), dissemination area (DA)-level income quintile (B, E), or DA-level proportion racialized quintile (C, F). 95% confidence intervals were estimated via bootstrap with 1,000 replicates. Announ., announcement of the vaccine passport; Implem., implementation of the vaccine passport.
Interrupted time series: vaccine passports’ impact on coverage by age
We estimated that, in the absence of the vaccine passports, first-dose vaccination coverage would have been 0.9 p.p. lower (95%CI: 0.4-1.2) in Québec by October 23rd and 0.7 p.p. lower (95%CI: 0.5-0.8) in Ontario by November 13th. In other words, vaccine passports led to relative increases of 23% (Québec; 95%CI: 10-36%) and 19% (Ontario; 95%CI: 15-22%) more vaccinations (Figure 2).
Observed (points) and modeled (blue and yellow) vaccine coverage over time is shown. Predicted vaccine coverage was obtained from three different regression models where the vaccination rate and the impact of the vaccine passport were allowed to vary by age group (A, D), DA-level income quintile (B, E), or DA-level proportion racialized quintile (C, F). Estimates and 95% confidence intervals (CIs) of the impact of the vaccine passport (observed coverage minus modeled counterfactual) are shown at the right of each panel. 95% CIs were estimated via bootstrap with 1,000 replicates. Announ., announcement of the vaccine passport; Implem., implementation of the vaccine passport.
The largest impact of the vaccine passport was observed in the 12-17 age group in Québec, where vaccine coverage was 2.3 p.p. higher (95%CI: 2.0-2.7) than it would have been in the absence of the vaccine passport. In Ontario, the corresponding impact was an increase of 1.3 p.p. (95%CI: 0.9-1.7). The smallest effects were estimated in the 60+ age group, where the impact was around 0.1 p.p. in both provinces (Figure 2A&D). Similar age patterns were observed for the relative impact of the passports (Supplementary Table S2). In Montréal and Toronto, the effect sizes for each age group (except for the 12-17 age group in Toronto) were equivalent to provincial estimates (Supplementary Figure S2). The observed age trends for the absolute impact remained when holding baseline vaccine coverage constant across DAs (Supplementary Figure S3).
Modification of vaccine passports’ impact on vaccine coverage by age and social determinants of health
When examining the impact by income quintile, we found little evidence of heterogeneity in Québec. In this province, the vaccine passport increased vaccine coverage in the lowest-income DAs by 1.1 p.p. (95%CI: 0.2-1.8) compared to 0.7 p.p. (95%CI: 0.4-0.8) in the highest-income DAs, corresponding to relative increases of 21% (95%CI: 4-40%) and 27% (95%CI: 15-36%), respectively (Figure 2B, Supplementary Table S2). When stratifying by age, the impact of vaccine passports was generally larger in lower-income DAs in most age groups (no clear trend in the 18-29 group). However, uncertainty was large and CIs overlapped across quintiles (Figure 3A).
The vaccine passport’s impact (defined as the observed vaccination coverage minus the modeled counterfactual coverage in the absence of a vaccine passport) was estimated from two different regression models where the vaccination rate and the impact of the vaccine passport were allowed to vary by the interaction of age and either DA-level income quintile (A, C), or DA-level proportion racialized quintile (B, D). 95% confidence intervals —in parenthesis—were estimated via bootstrap with 1,000 replicates. p.p., percentage points.
The impact was comparable across income quintiles in Ontario at around 0.7 to 0.8 p.p. (Figure 2D), although the relative increase in vaccinations ranged from 19% (95%CI: 10-29%) in the lowest-income DAs to 32% (95%CI: 25-40%) in the highest-income ones (Supplementary Table S2). The lack of heterogeneity in the absolute impact remained with age stratification — the estimated vaccine passport impact was larger in younger age groups but similar across income quintiles within each age group (Figure 3C).
For the proportion racialized, the impact was homogeneous at the DA-level. In Québec, the increase in vaccine coverage was around 0.7 to 1.0 p.p. across quintiles of proportion racialized, with no clear trend (Figure 2C). The relative impact ranged from increases in vaccination of 12% (95%CI: 7.5-18) in DAs with the highest proportion racialized to 29% (95%CI: 10-41%) in the lowest-proportion ones (Supplementary Table S2). Within age groups, the impact was larger in DAs with lower proportion racialized, except for the 12-17 age group where the impact was larger in higher-proportion DAs. Although CIs overlapped across some quintiles, uncertainty was smaller than in the income analyses (Figure 3B).
In Ontario, the DA-level impact was also similar regardless of the proportion of racialized residents. The absolute effect of vaccine passports was 0.7-0.8 p.p. in all quintiles and relative impacts were also homogeneous, ranging from increases of 19% to 24% (Figure 2F, Supplementary Table S2). As in Québec, there was more heterogeneity when stratifying by age, and the impact was bigger in DAs with lower proportion racialized. The effect was attenuated in older age groups, but the gradient remained in all age groups (Figure 3D).
The patterns by income and proportion racialized in Montréal and Toronto were equivalent to those of their respective provinces. One exception was the pattern in the vaccine passport impact by proportion racialized in Toronto, as there was a slight gradient only among people aged 12-29 years (Supplementary Figures S2&S4). When holding baseline vaccination coverage constant, the trends along SDOH remained for all cases except for income in Ontario, where there was a slight gradient in the impact of the vaccine passport (Supplementary Figure S5).
Sensitivity analyses
In sensitivity analyses (age model), changing the time-series start by ±1 week did not substantially change the estimated effect of the vaccine passport in Québec and slightly lowered it in Ontario (Supplementary Figure S6). In contrast, in models that assumed a different duration for the vaccine passports’ impact (5 or 7 weeks), the impact was slightly lower in both provinces, but model fit was also poorer using these specifications (Supplementary Figures S7-S8). Lastly, when using the best non-spline alternative model specifications (described in Supplementary Methods), the estimated impact of the vaccine passport was slightly higher in Québec and lower in Ontario, as compared to our spline-based approach. All impact estimates were higher when modelled using a simple log-linear model, but these methods had poorer fit (Supplementary Figure S9-S10).
Discussion
We found that vaccine passports increased COVID-19 first-dose vaccine coverage by approximately 1 p.p. in both Québec and Ontario, where first-dose vaccine coverage was above 80% (≥12-year-old population) at the time passports were announced. This translates to relative increases of 23% (Québec) and 19% (Ontario) more vaccinations among people without a first dose. The impact was largest among younger age groups (<40 years), even after controlling for baseline vaccine coverage. Differences in the impact of vaccine passports by area-level income or proportion of racialized residents were relatively small and the estimates’ uncertainty overlapped, suggesting that vaccine passports had limited impact on reducing socioeconomic disparities in vaccination coverage.
In both provinces, there were inequalities in the pre-announcement vaccination coverage by DA-level income. However, there was only a small gradient in the impact of the vaccine passport in Québec (i.e., higher impact in lower-income DAs), again with overlapping confidence intervals. In Ontario, there was little heterogeneity in the impact of vaccine passports by DA-level income. Taken together, these results suggest that vaccine passports may have led to larger gains in lower-income neighbourhoods in Québec, but not Ontario.
The fact that there were inequalities in baseline vaccine coverage by DA-level proportion of racialized residents in Québec, but not Ontario, could be attributed to different vaccination policies. Québec’s vaccine prioritization focused mostly on age and essential workers, whereas Ontario eventually implemented a “hotspot strategy,” which directed more vaccine-related resources to geographical areas with higher cumulative COVID-19 incidence — which on average had a higher proportion of racialized residents.38,39 Although estimates were uncertain, larger absolute effects were observed in neighbourhoods with lower proportions of racialized residents in age-stratified analyses in both provinces. This suggests that vaccine passports may have had slightly larger impacts in predominantly white neighbourhoods despite their higher baseline coverage — a heterogeneity that was masked by differences in age structure and that could manifest in increased disparities in lower-coverage settings.
Our effect size estimates are lower than those previously reported from Europe and Canada.12–14 For instance, two studies that evaluated vaccine passports in Italy, France, and Germany found absolute increases in vaccine coverage in the 5-13 p.p. range.13,14 In these three countries, however, passports were announced when the fraction of people without a first dose was much larger than Canada (30-35% instead of <20%). In Canada, a study reported slightly higher effects for vaccine passports in Québec (3.1 p.p.) and Ontario (1.9 p.p.).14 In contrast to our approach, the authors assumed that vaccine passports would have a permanent effect and positively impact vaccination rates beyond six weeks. Further, they did not account for the continuous reduction in size of the population without first dose, potentially overestimating the impact of vaccine passports.
Our results should also be interpreted by considering vaccine acceptance/hesitancy as a continuum between total acceptance and total refusal.40,41 First, vaccine passport policies may have had the biggest impact on those open to vaccination but for whom it was not a priority (i.e., who had scheduled it later in time or were considering it). This could partly explain the observed age effect: younger people may have decided to get vaccinated or moved their vaccinations forward in time to maintain access to non-essential settings and activities targeted by vaccine passports. Second, survey data suggest the majority of residents in Canada expressed positive attitudes toward COVID-19 vaccinations.42 Coupled with community-based efforts to improve engagement, increase awareness, and facilitate access (e.g., community ambassador programs, mobile vaccination clinics), there was a large “early adopter” effect by the time the vaccine passports were announced. The remainder of those not yet vaccinated by the time of the vaccine passport announcements may have largely comprised individuals experiencing long-standing, systemic, and persistent barriers to vaccination and/or vaccine mistrust. Our findings suggest that different strategies are needed to address these issues and increase vaccine acceptance and uptake in these communities.43
Various limitations should be considered when interpreting these results. First, interrupted time-series methods are sensitive to counterfactual specifications. However, our main results were generally robust to alternative model specifications. Second, concurrent events (e.g., return to school, university-/college-based mandates in Ontario, vaccine lottery in Québec) may have biased estimates of effectiveness upwards. However, school-related events would only partly affect age groups <30, and there is mixed evidence on the impact of vaccine lotteries for COVID-19 vaccination.44,45 Third, we used area-level measures of income and racialization, meaning that inferences on the role of individual-level income or racialization could be subject to ecological fallacy. Lastly, this study does not address other ways in which vaccine passports could affect the transmission SARS-CoV-2 (e.g., reduced mixing between people of different vaccination status in non-essential settings). Strengths of our study include the use of detailed DA-level information on vaccinations in Canada’s largest provinces. We also conducted a range of sensitivity analyses that provided credence to our estimates. Lastly, we investigated heterogeneity of impact by age and area-level social determinants of health — known drivers of inequalities in COVID-19 burden.
Conclusion
In Québec and Ontario, vaccine passports increased COVID-19 vaccination coverage, but absolute gains were small given the provinces already had relatively high vaccination coverage. The impact of vaccine passports was largest among younger age groups in both provinces. However, the effect of vaccine passports varied little by neighbourhood-level SDOH. Ultimately, other policies that account for how social determinants shape barriers to vaccination may be necessary to further increase vaccination coverage and meaningfully reduce inequities in COVID-19-related morbidity and mortality.
Data Availability
The analysis code is available at https://github.com/pop-health-mod/vaccine-passport-release. The authors had access to data from the vaccination registries used in this study under agreements with the Institut national de santé publique du Québec and Ontario's Ministry of Health. These data are not available for public release. The census data used in this study can be downloaded from Statistics Canada at https://www150.statcan.gc.ca/n1/en/catalogue/98-316-X2016001.
Declarations
Ethics approval
Ethics approvals were obtained from the Institutional Review Board of Faculty of Medicine and Health Sciences of McGill University in Québec (A06-M52-20B) and the Health Sciences Research Ethics Board of University of Toronto in Ontario (no. 39253).
Author contributions
MMG, SM, and JLFA designed the study. JLFA, HM, MAH, MMG, SM, and RB contributed to the conception of the analytical strategy. HM, MAH, and JLFA took care of data management. JLFA carried out all statistical analyses. YX, SH, DB, MB, MPH, KM, AK, RB, SB, and ED all provided substantial input to interpret the results. JLFA drafted the manuscript. All authors contributed to data interpretation and revision of the final manuscript.
Data availability
The analysis code is available at https://github.com/pop-health-mod/vaccine-passport-release.
The authors had access to data from the vaccination registries used in this study under agreements with the Institut national de santé publique du Québec and Ontario’s Ministry of Health. These data are not available for public release.
The census data used in this study can be downloaded from Statistics Canada at https://www150.statcan.gc.ca/n1/en/catalogue/98-316-X2016001.
Supplementary data
Supplementary methods and results are available online.
Funding
This study was funded by the Canadian Institutes of Health Research.
Conflict of interest
DB reports past contractual agreements with Institut national d’excellence en santé et en services sociaux (INESSS). MB reports grants from: Canadian Institutes of Health Research (CIHR), Medical Research Council, UK (MRC), the Bill and Melinda Gates Foundation, the Centers for Disease Control and Prevention (CDC), FRSQ/FCAR health research, the World Health Organization (WHO), the Public Health Agency of Canada (PHAC), the Québec Ministry of Health and Social Services, and the Institut national de santé publique du Québec (INSPQ). MMG reports a research grant from Gilead Sciences Inc., contractual agreements with the WHO and the Joint United Nations Programme on HIV/AIDS, and past contractual agreements with INESSS and INSPQ.
Acknowledgements
The authors would like to thank Geneviève Cadieux for her helpful feedback on previous versions of these analyses. MM-G research program is funded by a Canada Research Chair (Tier 2) in Population Health Modeling; and SM research program is funded by a Canada Research Chair (Tier 2) in Mathematical Modeling and Program Science.