ABSTRACT
Billions of doses of COVID-19 vaccines have been administered around the world, dramatically reducing SARS-CoV-2 incidence in some settings. Many studies suggest vaccines provide a high degree of protection against infection and disease, but precise estimates vary and studies differ in design, outcomes measured, dosing regime, location, and circulating virus strains. Here we conduct a systematic review of COVID-19 vaccines as of August 2021. We included efficacy data from Phase 3 clinical trials for 13 vaccines within the WHO Emergency Use Listing evaluation process and real-world effectiveness for 5 vaccines with observational studies meeting inclusion criteria. Vaccine metrics collected include effects against asymptomatic infection, any infection, symptomatic COVID-19, and severe outcomes including hospitalization and death, for both partial and complete vaccination, and against SARS-CoV-2 variants of concern. In addition, we review the epidemiological principles behind the design and interpretation of vaccine effects and explain important sources of heterogeneity between studies.
Introduction
On March 11, 2020, the World Health Organization (WHO) declared SARS-CoV-2 a global pandemic1. Worldwide, over 200 million COVID-19 cases and over 4 million deaths have been recorded to date. SARS-CoV-2’s rapid global spread and its alarming clinical severity have accelerated demand for COVID-19 immunizations that safely and effectively prevent disease incidence or reduce severity. Despite the traditionally prolonged vaccine development timeline, over 20 COVID-19 vaccine candidates have received emergency use authorization in at least one country, and 3 billion people have been vaccinated globally2,3.
Evidence from clinical trials and observational studies overwhelmingly support the safety and immunogenicity of numerous COVID-19 vaccines, especially when it comes to protection against severe infection and death in fully vaccinated individuals. However, precise estimates of vaccine efficacy and effectiveness (“VE”) have varied across studies due to a range of factors. For example, an interim analysis across four Phase 3 trial sites found AstraZeneca’s two-dose viral vector vaccine (AZD1222) had 70% efficacy against symptomatic COVID-19 disease; when disaggregated by dosing schedule, estimated VE was nearly 30% higher in the sub-cohort receiving a modified low vaccine dose followed by a standard dose (90%) compared to the sub-cohort receiving two standard doses (62%)4. Observational studies in the UK5–8, Scotland9, Brazil10, and the Netherlands11, have since presented a range of VE estimates (60% to 94%) obtained from non-randomized designs (i.e., test-negative control, prospective cohort) using effectiveness measures (e.g., any SARS-CoV-2 infection) differing from the outcomes of the clinical trials.
SARS-CoV-2 evolutionary dynamics have further confounded interpretation of heterogeneous VE estimates obtained from comparably designed COVID-19 vaccine studies, particularly under real-world, non-experimental conditions. The emergence of SARS-CoV-2 strains designated variants of concern (VOC) by WHO, including Alpha (B.1.1.7), Beta (B.1.351), Gamma (P.1), and Delta (B.1.617.2), has recently increased concern over attenuated VE given that some variants are associated with higher viral load12–16 and evasion of neutralizing antibodies ex vivo17–19. Recent studies have reported diminished effectiveness of the Pfizer-BioNTech BNT162b2 vaccine against the Delta SARS-CoV-2 variant20,21, relative to the 95% VE measure against parent SARS-CoV-2 lineages obtained from clinical trials22. Efforts to attribute these observed reductions to waning immunity, diminished protection against circulating SARS-CoV-2 variants, study methodology, or other contextual factors (i.e., presence vs absence of non-pharmaceutical interventions) is constrained by their co-occurrence and endogeneity. Nonetheless, synthesizing the totality of the evidence for COVID-19 VE, and understanding the constellation of factors contributing to observed heterogeneities, is imperative for policy-makers to design effective and equitable COVID-19 vaccination campaigns during a rapidly evolving pandemic.
Here, we systematically reviewed vaccine efficacy and effectiveness data for COVID-19 vaccines against various clinical outcomes, specifically asymptomatic infection, any infection, symptomatic disease, hospitalization, and death. Additionally, our review considers VE measures for both full and partial immunization courses and circulating SARS-CoV-2 variants of concern.
Methods
For the purpose of this study, we use the abbreviation “VE” to refer to both vaccine efficacy (from randomized clinical trials) and vaccine effectiveness (from observational studies). We chose to review all vaccines which had received, or submitted applications for, Emergency Use Listing from WHO as of August 15, 202123 and had at minimum publicly released data from completed Phase 3 trials (Table 1). We searched for clinical trial efficacy results published in peer-reviewed scientific journals (PubMed, Google Scholar), preprint servers (medRxiv, bioRxiv), government public health agency websites, in news articles (Google), on the vaccine manufacturers’ websites, and in the databases of medical regulatory agencies (e.g., the US Food & Drug Administration or the European Medicines Agency). Searches were conducted using the vaccine’s brand, trade, or research name. To locate observational studies of vaccine effectiveness, we used a detailed search query applied to multiple databases (Supplementary Methods), and only included results that appeared in at least a detailed report or preprint form. Studies underwent an initial review of the title/abstract, before progressing to a more detailed full-text review (Figure S1). From each document, we extracted VE against any stage of infection (Table S1). For the full vaccine course (1 or 2 doses, depending on the vaccine brand), results were only included if at least 1 week had passed between the final dose and case detection. For VE after a partial course, cases must have occurred at least 2 weeks after the first dose but before the second dose.
Vaccine products are listed in descending order based on the number of countries in which the vaccine is currently in use. Vaccines included in the current study - based on availability of efficacy data from Phase 3 clinical trials - are highlighted in grey. Abbreviations: EUL = emergency use listing, EOI = expression of interest, CI = confidence interval. Vaccine details were obtained from McGill University’s COVID-19 Vaccine Tracker: https://covid19.trackvaccines.org/, which aggregates data from multiple sources. Original source for WHO EUL status : https://extranet.who.int/pqweb/key-resources/documents/status-covid-19-vaccines-within-who-eulpq-evaluation-process
Observational studies were excluded if a proper control group was not used (e.g., modeled or historic controls), outcomes were not laboratory-confirmed, the study design did not attempt to account for confounding, vaccination status was determined by self-report (not documented) for >10% of participants, confidence intervals (CIs) were not reported (except in cases where it was not possible to calculate CIs), significant bias was present as determined by expert opinion, if the early post-vaccination period was used as the reference period for calculating VE (e.g., day 0-12 vs. day 12-21), or if the definition of unvaccinated included days 0-12 post vaccination. For this review only studies including persons with and without the clinical outcome under investigation and with and without vaccination were included. Thus, impact studies and studies that only evaluated the risk of progression to severe disease among SARS-CoV-2-positive individuals were excluded. We evaluated studies that reported VE for a combination of vaccines but did not include those values here unless they also disaggregated results by vaccine product. Finally, we excluded studies that only presented 1st dose VE of a 2-dose vaccine while including some persons who had received 2 doses in the estimate. We classified VE as specific against a particular SARS-CoV-2 variant if sequencing or other molecular detection methods were used within the study to confirm the variant causing infection in all individual cases contributing to a VE estimate. The data from observational studies included here is also available in table and graphical formats on VIEW-hub, a resource developed by the International Vaccine Access Center at Johns Hopkins University (view-hub.org/resources) and in the Data Supplement (Table S1).
Defining how well a vaccine prevents infection and disease
Like other pharmaceutical products, vaccines are evaluated in clinical trials for both safety and efficacy, and in this review, we focus on the latter. Vaccine efficacy is defined as the amount by which vaccination reduces the probability that an individual develops disease in a particular time period compared to those who did not receive the vaccine. It is calculated using this formula:
where sometimes the denominator “# vaccinated” (“# unvaccinated”) is replaced with the sum of the total time enrolled in the study among vaccinated (unvaccinated) subjects (i.e., the “person time”)24,25.
Vaccine efficacy only describes the relative, as opposed to absolute, risk of disease. For example, if the risk of disease within a certain time frame is reduced from 50% in unvaccinated individuals to 10% in vaccinated individuals, then the vaccine efficacy (80%) is the same as in another setting where the risk of disease in that time frame was reduced from 5% among the unvaccinated to 1% among the vaccinated. This is a desired feature of a metric for vaccine strength, since absolute risk may change over time during an epidemic, due to factors like seasonality and non-pharmaceutical interventions (e.g., shelter-in-place ordinances, face mask use). The value of vaccine efficacy also does not typically tell us how an imperfect vaccine fails. A 90% efficacious vaccine could mean one of three things: (1) that 90% of vaccinated individuals are completely resistant to disease while the other 10% are as susceptible as unvaccinated individuals (“all-or-nothing”), (2) that all vaccinated individuals have exactly 1/10th the risk of getting infected from any given exposure (i.e., a “leaky” vaccine), or (3) a combination of these scenarios25–27.
To measure vaccine efficacy for COVID-19 and in general, the specific clinical outcome that the vaccine is meant to prevent (the definition of “disease”) must be carefully defined. The ideal goal of vaccination is to completely prevent infection, meaning that the vaccine-induced immune response must be able to block the earliest attempts of the pathogen to replicate within an individual’s body. If infection cannot be established, then this individual will not experience symptoms of the disease nor transmit to susceptible individuals. This sort of “sterilizing immunity” is rare, and vaccine efficacy against infection is difficult to measure in practice28,29 since, for short-lived and commonly asymptomatic infections like SARS-CoV-2, this would require frequent testing of everyone in the study population. Moreover, high efficacy against infection is not necessary for a vaccine to be beneficial. To reduce the public health impact of an infectious disease, it might be enough to prevent the symptoms of the disease, even if infection still occurs30. Even vaccines that do not completely prevent infection may trigger immune responses that can reduce viral load, the duration of infectiousness, or spread of virus between tissues, preventing deleterious clinical outcomes29–31.
For COVID-19, the primary vaccine efficacy outcome chosen as the endpoint for most clinical trials was the ability to prevent symptomatic, laboratory-confirmed COVID-19 disease30. This outcome was defined as the occurrence of COVID-19-associated symptoms (e.g., cough, shortness of breath, fever) occurring in the presence of detectable SARS-CoV-2. This definition of vaccine efficacy represents a trade-off between practicality and public health importance. Symptoms can be self-assessed by participants and testing restricted to those reporting symptoms. However, most individuals with COVID-19 recover completely with only mild or moderate symptoms, and the major concern is the subset of individuals who develop more severe disease32–34. While the fraction of infections progressing to a severe stage is high enough to have overwhelmed healthcare resources globally and led to millions of excess deaths in 2020 and 2021, it is still rare enough that few events would be expected to occur in a clinical trial of tens of thousands of individuals lasting a few months. While many trials did report efficacy against severe outcomes like hospitalization or death as secondary outcomes despite small numbers22,35–44, non-randomized post-approval studies provide further estimates of these metrics for much larger sample sizes. In this paper, we summarize vaccine efficacy and effectiveness values reported for all stages of infection from both clinical trials and observational studies.
Randomized and observational studies for estimating vaccine effects
Formally, the calculation of VE compares the likelihood of infection in vaccinated individuals to the hypothetical risk they would have had if they were unvaccinated. In reality, hypothetical comparisons are impossible and must be made between different groups of individuals. Vaccine studies therefore strive to choose comparable groups in terms of disease risk, so any observed differences between them can be attributed to vaccination. One approach is to conduct a study where subjects are randomly assigned to receive the vaccine or not, which, with large enough sample sizes usually ensures there will not be any significant differences. In reality, randomization is not always possible, so “observational” studies use data from situations where vaccine administration was non-random and attempt to isolate the effect of vaccination by accounting for differences between groups in the study design and analysis.
This review includes data from both randomized controlled trials (RCTs) that estimate vaccine efficacy and observational studies that report real-world vaccine effectiveness, which each have pros and cons45,46. The most obvious strength of RCTs is that randomization helps ensure that the results are not biased by participants’ health-seeking behaviors and risk factors for disease. While RCTs are the gold-standard for vaccine studies30, they are very costly and rarely include more than tens of thousands of participants. Consequently, the incidence of more severe outcomes of infection is often limited and long study durations may be needed. RCTs may exclude participants with a high risk of death, or populations who need the vaccine but who are too risky to include in the trial, such as pregnant women, young children, and those with comorbid health conditions. The age and race composition of trial populations may also be relatively homogenous. Finally, participants in an RCT may become unblinded if they are told their vaccination status by study administrators or infer it based on vaccine side effects, which can introduce bias or confounding into the study.
Once a vaccine is shown to be safe and efficacious in clinical trials and is authorized for general use, further randomized trials are often considered unethical or impractical, especially in the setting of a wide-spread epidemic like the COVID-19 pandemic. Instead, observational studies are used to augment estimates of vaccine efficacy with values of real-world effectiveness and include designs such as case-control studies (including test-negative designs) and cohort studies (prospective or retrospective)25,47,48. Compared to RCTs, observational studies have their own costs and benefits. They must carefully address biases due to behavioral differences in those who chose to or were eligible to receive vaccines in real-world settings compared to those who did not. For example, vaccine recipients may be more cautious and have fewer possible exposures than individuals who chose not to be vaccinated; they may also be less cautious if they believe they are protected by vaccination. On the other hand, observational studies provide a more realistic picture of population heterogeneity compared to RCTs, and likely include populations of interest, like people who are immunocompromised. During massive national vaccination campaigns like those occurring for COVID-19, it is also possible to obtain much larger sample sizes from real-world observational studies, therefore arguably improving the precision of estimates for effectiveness, especially against severe outcomes.
Sources of heterogeneity across studies
COVID-19 vaccine studies were conducted by many independent research teams and in diverse epidemic settings around the world (Table 1, Table 2, Table S1, Figure 1). Consequently, there are several potential sources of heterogeneity between studies that make it difficult to compare VE estimates between them. These factors have been described elsewhere48,49 and include:
Study population: When a study includes a greater number of participants who are at higher risk of developing symptomatic disease (e.g., older individuals) or with conditions that could reduce the immunogenicity of vaccines (e.g., people living with HIV), VE can be reduced.
Outcome and case definition: VE values differ between disease outcomes with varying levels of severity. Even when studies have the same stated outcome, the case definition can vary substantially. For example, most clinical trials used ‘symptomatic disease’ with laboratory confirmed SARS-CoV-2 infection as the primary outcome but included anywhere from 5 (for AstraZeneca/AZD122250) to 16 (Janssen/Ad26.COV2.S36) different potential symptoms and varied in requiring one or two to be present. These differences are exacerbated in effectiveness studies that often rely on passive surveillance by health systems. In addition, differences in the timing included in the definition of the disease outcome (e.g., death within 30 days after diagnosis) can lead to heterogeneities.
Follow-up period: Since it takes time after vaccination for an effective adaptive immune response to develop, studies that begin the estimation of VE sooner could observe reduced VE (e.g., 7+ days for Novavax/NVX-CoV2373 vs 28+ days for Janssen/Ad26.COV2.S). In addition, immune protection may eventually start to wane over time, and so studies that cover longer periods from time since vaccination could also lead to a lower VE.
Predominant variants: Some SARS-CoV-2 variants of concern have been observed to exhibit immune-escape properties17 (e.g., Beta51–53 and Delta20,21,35 variants). Studies conducted when such variants account for a greater proportion of overall infections could result in lower observed vaccine efficacy (Figure 1).
Force of infection: In studies done during time periods of higher prevalence of circulating virus (Figure 1), the number of exposures each individual experiences during the study period could be increased, which could make it more likely at least one exposure overwhelms vaccine-induced immunity and could lead to reduced observed VE, especially for vaccines that reduce per exposure risk (“leaky” vaccines)25,54.
Study design and analysis: Randomization in vaccine trials can reduce (but not eliminate) confounding. In the absence of randomization, vaccine effectiveness studies can attempt to reduce confounding through study design (e.g., the test-negative design) and during the analysis (e.g., controlling for potential confounding variables in regression models). Some studies, especially those that use administrative data, may not collect and therefore control for such possible confounders.
Number of studies of each type included in the review. No studies reported for other vaccine candidates met our inclusion criteria (see Methods). Details of all the individual studies are included in Table S1.
For each country, the time period during which outcomes were observed during each vaccine trial is shaded grey. For each two week period, the average daily incidence of reported cases is shown (height of bars)147. The contribution of each major variant of concern to total case counts is estimated from the reported fraction of sequenced SARS-CoV-2 samples belonging to that strain (fill color)148. Figure includes only vaccine trials described in published or pre-print reports, trial sites with at least 10,000 individuals from the general adult population, and countries regularly reporting SARS-CoV-2 lineages to the GISAID database.
Results of COVID-19 vaccine studies
As of August 15, 2021 there were 24 unique vaccine products that had entered the WHO Emergency Use Listing evaluation process, six of which had received authorization. The products under evaluation included mRNA, viral vector, inactivated virus, protein subunit, and conjugate vaccines, and were developed by a mix of pharmaceutical companies, non-profit research institutes, and government agencies (Table 1). Thirteen of these vaccine candidates completed Phase 3 clinical trials and released their VE estimates and uncertainty intervals publicly (Figure 2), which we collected via 21 separate reports 22,35–44,50–52,55–60(Figure S1). This included Oxford/AstraZeneca’s AZD1222, Bharat Biotech’s BBV152, BioCubaFarma’s Abdala, Soberano 2, and Soberano 2 Plus, Gamaleya Institute’s Sputnik V, Janssen’s Ad26.COV2.S, Moderna’s mRNA-1273, Pfizer/BioNTech’s BNT162b2, Sinopharm Beijing’s BBIBP-CorV, Sinopharm Wuhan’s WIBP-CorV, and Sinovac’s CoronaVac. Vaccine efficacy against symptomatic COVID-19, the primary outcome of each trial, ranged from 66-95%. Vaccines with the highest reported efficacy included both first-in-class mRNA vaccines as well as protein subunit vaccines, a fifty year-old technology. Some of these clinical trials additionally reported efficacy against any infection or against more severe forms of disease (included in Figures 3-7, Table S1), though confidence intervals on the latter tended to be very large due to limited sample sizes and trial durations.
Each efficacy value is for the complete vaccine course (1 dose for Ad26.COV2.S/Janssen, 3 doses for BioCubaFarma/Abdala and Soberana02+/BioCubaFarma, and 2 doses for all others).
Estimates are colored by the viral variant against which the VE value was measured. Solid markers are estimates from randomized clinical trials (efficacy values), and open markers are estimates from observational studies (effectiveness values). The source of each estimate is given by the labels on the left side (“(reference number) Country, population”). Within each disease severity level, estimates are ordered alphabetically by country, and then by population
Estimates are colored by the viral variant against which the VE value was measured. Solid markers are estimates from randomized clinical trials (efficacy values), and open markers are estimates from observational studies (effectiveness values). The source of each estimate is given by the labels on the left side (“(reference number) Country, population”). Within each disease severity level, estimates are ordered alphabetically by country, and then by population.
Estimates are colored by the viral variant against which the VE value was measured. Solid markers are estimates from randomized clinical trials (efficacy values), and open markers are estimates from observational studies (effectiveness values). The source of each estimate is given by the labels on the left side (“(reference number) Country, population”). Within each disease severity level, estimates are ordered alphabetically by country, and then by population.
Estimates are colored by the viral variant against which the VE value was measured. Solid markers are estimates from randomized clinical trials (efficacy values), and open markers are estimates from observational studies (effectiveness values). The source of each estimate is given by the labels on the left side (“(reference number) Country, population”). Within each disease severity level, estimates are ordered alphabetically by country, and then by population.
Estimates are colored by the viral variant against which the VE value was measured. Solid markers are estimates from randomized clinical trials (efficacy values), and open markers are estimates from observational studies (effectiveness values). The source of each estimate is given by the labels on the left side (“(reference number) Country, population”). Within each disease severity level, estimates are ordered alphabetically by country, and then by population.
Estimates are colored by the vaccine for which the VE value was measured. Solid markers are estimates from randomized clinical trials (efficacy values), and open markers are estimates from observational studies (effectiveness values). The source of each estimate is given by the labels on the left side (“(reference number) Country, population”). Within each disease severity level, estimates are ordered alphabetically by country, and then by population.
We identified a total of 58 real-world vaccine effectiveness studies covering five vaccines that met our inclusion criteria - BNT162b2 (Pfizer-BioNTech), mRNA-1273 (Moderna), Ad26.COV2.S (Janssen/Johnson & Johnson), ChAdOx1 nCoV-19 (Oxford-AstraZeneca), and CoronaVac (Sinovac) (Table 2, Figure S1, Table S1)5–11,21,53,61–107. These observational studies provided VE metrics that were not available or estimated with high uncertainty in clinical trials, such as effectiveness against more severe outcomes, against any SARS-CoV-2 infection, against asymptomatic infection, against specific circulating SARS-CoV-2 variants of concern, and after only a single dose of two-dose vaccine courses (Figures 3-7). In general, effectiveness estimates were high for full vaccine courses and overlapped with efficacy values. Vaccines are more efficacious at preventing severe infection or death compared to symptomatic COVID-19. The degree to which the vaccine prevented any infection, and the degree to which partial courses prevented infection or disease, varied significantly by product. Data was especially sparse for VE against death or asymptomatic infection and against the Gamma (P.1) variant.
The most data was available for BNT162b2, the two-dose mRNA vaccine developed by Pfizer/BioNTech, mainly due to its use very early in 2021 in Israel, the UK, and the US (Figure 3). Overall VE estimates measured after both doses in the general population for all study types ranged from 90-100% for death, 85-98% for severe infection, 80-95% for symptomatic disease, 65-95% for any infection, and 65-90% for asymptomatic infection. These values were lower after only a single dose: 60-90% for severe infection, 30-90% for symptomatic disease, and 20-90% for any infection. Where comparisons were possible, effectiveness values from observational studies overlapped with efficacy estimates from clinical trials. Some studies found lower VE in special populations, including residents of long-term care facilities53,92 and the elderly91. Heterogeneities between studies made direct comparisons of VE for variants of concern difficult. A few studies with head-to-head comparisons suggested reduced VE for BTN162b2 against the Beta (B.1.351) and Delta (B.1.617.2) compared to Alpha (B.1.1.7) variants of concern for symptomatic cases or any infection20,21,61,89.
The other authorized two-dose mRNA vaccine, Moderna’s mRNA-1273, was also relatively well-studied based largely on data from the USA, Canada, and Qatar (Figure 4). After both doses, overall VE estimates in the general population fell between 90-96% for severe disease, 87-100% for symptomatic disease, 85-98% for any infection, and 92% for asymptomatic infection. With only a single dose VE values were 75-80% for severe disease, 55-92% for symptomatic disease, 30-90% for any infection, and 45-60% for asymptomatic infection. No studies that met our inclusion criteria reported VE against death. Studies in Qatar20,70 estimated VE for symptomatic infection with Delta after both doses compared to overall estimates or estimates for earlier variants (e.g. Alpha), and found reductions of ∼10%. Similar reductions were not seen for VE against more severe outcomes or for the Alpha variant.
AstraZeneca’s two-dose viral vector vaccine (AZD1222) was also the focus of many studies, especially after only a single dose, likely due to the fact that the recommended interval between doses is longer than other vaccines (12 weeks) and some countries including the UK and Canada adopted the strategy of prioritizing first doses over second doses in early 2021 (Figure 5). VE after both doses was 75-100% for severe disease, 65-80% for symptomatic disease, 60-85% for any infection, and 15% for asymptomatic infection overall in the general population. After only a single dose, VE ranges were 85-95% for severe disease, 50-75% for symptomatic disease, and 0-95% for any infection. There was no evidence that the Alpha variant led to a reduction in VE for AZD1222, whereas the Delta variant appeared to lead to a 10-15% reduction against symptomatic disease or infection after both doses and ∼20% after a single dose in some 21,89 but not all93 studies. Evidence from the clinical trial site in South Africa51 suggested loss of efficacy against any symptomatic infection with the Beta variant, but with much uncertainty (VE 10% CI [<0, 55]).
Effectiveness studies of Sinovac’s inactivated virus vaccine CoronaVac were only available from Chile85 and for individuals >70 years old in Brazil100, which complemented results from three separate clinical trials, including one restricted to health care workers (Figure 6). After both doses, in the general population, VE was 80-86% against death, 85-100% against severe disease, 65-85% against symptomatic disease, and ∼66% against any infection. With only one dose, these values were significantly reduced, to 40-46% for death, 37% for severe disease, and 16% for symptomatic disease and infection. No VE estimates for specific variants were available from any study, though the study in Brazil100 was conducted during a time when Gamma (P.1) was predominant, which could in part explain - along with the older study population - the lower VE estimates in this study.
Data was similarly sparse for Janssen/Johnson & Johnson’s Ad26.COV2.S single-dose viral vector vaccine, with clinical trials providing the majority of data (Figure 7). Overall VE estimates were 100% against death, 85% for severe disease, 66% for symptomatic disease, 12-77% for any infection, and 65% for asymptomatic infection. VE against infection with the Beta (B.1.351) variant was not significantly reduced during the clinical trial in South Africa36.
Although at the time of writing there were at least 12 other COVID-19 vaccines that had received emergency authorization for widespread use in at least one country, we could not locate effectiveness studies meeting our inclusion criteria. This was especially notable for Gamaleya’s Sputnik V and Sinopharm-Beijing’s BBIBP-CorV, which have each been deployed in dozens of countries (Table 1).
Discussion
The development of COVID-19 vaccines has been an astounding feat of science. Within a year of detecting the first outbreak and isolating the SARS-CoV-2 virus, multiple vaccines were being deployed around the world. In this review, we systematically collected and reported efficacy and effectiveness (“VE”) values by vaccine platform, disease outcome, number of doses, and SARS-CoV-2 variant. These findings demonstrated robust evidence for the high VE of COVID-19 vaccines both in clinical trials and real-world settings. We found that across all vaccine platforms, protection against severe infection or death in the general population was at least 80% and often close to 100%. VE against symptomatic disease was heterogeneous between vaccine products and studies but was almost always greater than 65% and often greater than 90%. The vast majority of studies showed that vaccines provided protection against infection itself - not just disease - demonstrating the potential for indirect protective effects (i.e., “herd immunity”). The degree of protection offered by only a single dose of two-dose vaccine courses varied by product. Most vaccines retained high levels of protection for most SARS-CoV-2 variants of concern, especially against severe outcomes. A few studies provided evidence of slight reductions in VE for infection or mild disease with the Beta (B.1.351) and Delta (B.1.617.2) strains. No studies meeting our inclusion criteria provided VE estimates for the Gamma (P.1) strain.
There are several important components of COVID-19 vaccine efficacy/effectiveness that we did not address in this study. Individuals previously infected with COVID-19 tend to have elevated levels of neutralizing antibodies (nAb) against SARS-CoV-2, and some studies have demonstrated that with only a single vaccine dose these individuals tend to reach similar nAb levels as individuals without a history of infection reach only after two doses108–110. Thus, VE could differ by serostatus. Some clinical trials excluded individuals seropositive at baseline and those that did not generally did not have enough power to test for differences in VE by serostatus at baseline. In Qatar, one observational study found that the vaccine effectiveness of BNT162b2 (Pfizer) was increased in individuals with a previous history of infection111, and another found that among unvaccinated persons, prior infection reduced the risk of new infection by 74%112. Because infection results in natural immunity, including persons previously infected with SARS-CoV-2 in the unvaccinated group can bias estimates of VE downward. Of 58 observational studies included in this review, 19 included persons with previous SARS-CoV-2 infection.
Knowledge gained from studying vaccines for other pathogens in humans and animals shows that time may be an important dimension for vaccine efficacy. After vaccination (or infection), there is a necessary period during which antibody-producing plasma cells and their B cell precursors expand to levels necessary to reduce the risk of infection or disease after an exposure113. This interval is often estimated as around 2 weeks, which is why studies almost always count only cases after this time period when calculating VE29,114,115. However, protection can continue to increase for a few months, as suggested in the Janssen (Ad26.COV2.S) trial36. Over longer time periods, immune protection is maintained by long-lived plasma cells and memory B and T cells, but numbers of these cells can decay over time, leading to waning in VE29,31,113,114,116. While most studies reviewed here were conducted in a short time period after vaccination (< 6 months) and did not examine waning, this is currently a critical issue for COVID-19 vaccines, and new studies will help address this question over the next year. Several recent reports have suggested reductions in VE over time16,41,117–120, while the WHO regards current evidence as inconclusive and stresses prioritization of primary doses121. Despite this, several governments have already begun administering additional “booster” doses (e.g., France122, Israel123), and others have announced tentative plans to do so in the future (e.g., US124).
For multi-dose vaccine regimens, the time interval between the initial dose and any follow up doses may affect the overall VE. While few clinical trials included variation in dose spacing, the trial of AZD1222 (AstraZeneca) varied the interval between doses and found that efficacy increased from 55% with less than 6 weeks between doses to 81% with more than 12 weeks50. In real-world settings, this interval often varied due to explicit policies of delaying second doses in favor of universal partial vaccination125,126, allowing for evaluation of its impact on both nAb levels and VE7,16,127. Currently, while most vaccine manufacturers and international advisory committees (including WHO128) recommend that initial and subsequent vaccine doses be with the same product - as this was the regimen tested in clinical trials - many countries have adopted more flexible policies allowing for “mixing and matching” of doses129–131, often in response to supply constraints. In immunology, this strategy is termed “heterologous prime boost” 132 and has been explicitly investigated in vaccine studies as a way of enhancing the strength and breadth of immunity (e.g., for HIV133, influenza134, and Ebola135). Gamaleya’s Sputnik V vaccine uses two different modified adenoviruses to deliver the gene for the SARS-CoV-2 spike protein in the first (Ad26) vs second doses (Ad5)42. Preliminary studies have shown that boosting AstraZeneca’s AZD1222 vaccine with Pfizer’s BNT162b2 or Moderna’s mRNA-1273 increases nAb levels compared to two doses of AZD1222 alone136–140, and an observational study in Denmark found an effectiveness of 88% against SARS-CoV-2 infection for the combination of 1 dose of AZD1222 and a second dose of either mRNA vaccine80. Further work is needed to understand which vaccine combinations are safe and effective for COVID-19.
At an individual level, the goal of vaccination against COVID-19 is to prevent morbidity (e.g. symptoms, hospitalization, death), but from a population-level perspective, there is an additional goal of reducing transmission. The additional indirect protection offered by vaccines slows the spread of infection, and removes the need for complete vaccine coverage, which is currently a challenge as no vaccine is approved yet for use in children. For individuals who become infected with SARS-CoV-2 despite vaccination (i.e., “breakthrough” infection), an important additional efficacy metric is the reduction in their potential for transmission141. A few studies have estimated this reduction for COVID-19 by enrolling and testing close contacts of cases11,87,96,142–145. SARS-CoV-2 viral load in the respiratory tract is expected to be a determinant of transmission risk, and other studies have measured reductions in viral load in breakthrough (versus unvaccinated) cases16,101,107,146. The ability of COVID-19 vaccines to reduce transmission, especially in individuals with asymptomatic or mild infection, is expected to be especially important as vaccine access is expanded to children, in whom infection tends to be mild.
In conclusion, data from a wide variety of study types and settings demonstrate that COVID-19 vaccines provide high levels of protection against severe disease, and additionally protect against infection and mild disease, even for major SARS-CoV-2 variants of concern.
Data Availability
All VE metrics reported in the review are provided in the data supplement (Supplementary Table 1)
AUTHOR CONTRIBUTIONS
ALH proposed the Review with input from BW, CBJ, JGR, MDK, MMH, and SAT. AAN, ALH, BW, CBJ, and JGR and conducted the search for clinical trials of COVID-19 vaccines and extracted efficacy values. AB, PSZ, KW and MMH conducted the search for observational studies of COVID-19 vaccines and extracted effectiveness values. MDK, DRF, and MKP supervised the collection of vaccine effectiveness studies including designing the search strategy, choosing the inclusion criteria, and evaluating studies against these criteria. ALH and MMH synthesized and interpreted the data on efficacy and effectiveness. BW assembled data on clinical trial timing, location, and SARS-CoV-2 variant prevalence. ALH, BW, and MMH created the tables and figures. ALH, CBJ, BW, JGR and MHH drafted the manuscript. All authors revised the manuscript and approved it for submission.
DECLARATION OF INTERESTS
Study authors have received grant funding from the Bill & Melinda Gates Foundation (MMH, MDK, ALH, AAN), US National Institutes of Health (ALH), the US Centers for Disease Control and Prevention (SAT), and the US Department of Health and Human Services (ALH, SAT) for work unrelated to this project. CBJ and JGR received funding from the Novel Coronavirus Research Compendium at Johns Hopkins to conduct reviews of COVID-19 vaccine papers for other purposes. MMH and MDK have previously received support from a grant from Pfizer Inc to Johns Hopkins University for a non-COVID-19 vaccine. BW provided unpaid technical support to Bharat Biotech related to the clinical development of the BBV152 vaccine candidate. DRF previously served on an independent data monitoring committee for GlaxoSmithKline for a non-COVID-19 vaccine candidate. SAT served as an expert consultant for Milliman, Inc on future COVID-19 trajectories. All other authors declare no competing interests.
ACKNOWLEDGMENTS
The authors thank M Kate Grabowski and the Johns Hopkins University Novel Coronavirus Research Compendium for bringing the study authors together to work on this paper. MHH, AB, PS, KW, and MDK received funding to collect the data used in work through a contract from the WHO to the International Vaccine Access Center at Johns Hopkins University. ALH and AAN received support from the US National Institutes of Health (NIH DP5OD019851). All other authors received no specific funding support for this work. The study sponsors had no role in the study design, in the collection, analysis, and interpretation of data, in the writing of the report, or in the decision to submit the paper for publication. The views represented in this article do not necessarily reflect the views of the WHO or the NIH.
REFERENCES
