Abstract
Importance Active monitoring of health outcomes following COVID-19 vaccination offers early detection of rare outcomes that may not be identified in pre-licensure trials.
Objective To conduct near-real time monitoring of health outcomes following BNT162b2 COVID-19 vaccination in the U.S. pediatric population aged 5-17 years.
Design We conducted rapid cycle analysis of 20 pre-specified health outcomes, 13 of which underwent sequential testing and 7 of which were monitored descriptively within a cohort of vaccinated individuals. We tested for increased risk of each health outcome following vaccination compared to a historical baseline, while adjusting for repeated looks at the data as well as claims processing delay.
Setting This is a population-based study in three large commercial claims databases conducted under the U.S. FDA public health surveillance mandate.
Participants The study included over 3 million enrollees aged 5-17 years with BNT162b2 COVID-19 vaccination through mid-2022 in three commercial claims databases. We required continuous enrollment in a medical health insurance plan from the start of an outcome-specific clean window to the COVID-19 vaccination.
Exposure Exposure was defined as receipt of a BNT162b2 COVID-19 vaccine dose. The primary analysis assessed primary series doses together (Dose 1 + Dose 2), and dose-specific secondary analyses were conducted. Follow up time was censored for death, disenrollment, end of risk window, end of study period, or a subsequent vaccine dose.
Main Outcome(s) and Measure(s) We monitored 20 pre-specified health outcomes. We performed descriptive monitoring for all outcomes and sequential testing for 13 outcomes.
Results Among 13 health outcomes evaluated by sequential testing, 12 did not meet the threshold for a statistical signal in any of the three databases. In our primary analysis, myocarditis/pericarditis signaled following primary series vaccination with BNT162b2 in ages 12-17 years across all three databases.
Conclusions and Relevance Consistent with published literature, our near-real time monitoring identified a signal for only myocarditis/pericarditis following BNT162b2 COVID-19 vaccination in children aged 12-17 years. This method is intended for early detection of safety signals. Our results are reassuring of the safety of the vaccine, and the potential benefits of vaccination outweigh the risks.
Question Did active monitoring detect potentially elevated risk of health outcomes following BNT162b2 COVID-19 vaccination in the U.S. pediatric population aged 5-17 years?
Findings Twelve of 13 health outcomes did not meet the safety signal threshold following BNT162b2 COVID-19 vaccination in three large commercial claims databases using near real-time monitoring. Myocarditis/pericarditis met the statistical threshold for a signal following primary series vaccination in ages 12-17 years.
Meaning Results from near-real time monitoring of health outcomes following BNT162b2 COVID-19 vaccination provide additional reassuring evidence of vaccine safety in the pediatric population. The myocarditis/pericarditis signal is consistent with current evidence and is being further evaluated.
Introduction
Three vaccines are currently available in the United States to prevent Coronavirus Disease 2019 (COVID-19) in children including Pfizer-BioNTech (BNT162b2), Moderna (mRNA-1273) for ages 6 months - 17 years, and Novavax (NVX-CoV2373) COVID-19 Vaccine ages 12-17 years.1-3 We present safety results from near-real time monitoring using commercial claims databases of over 3 million children aged 5-17 years following administration of the COVID-19 BNT162b2 vaccine, which was the first approved vaccine for the pediatric population. This study was conducted under the U.S. Food and Drug Administration (FDA) Biologics Effectiveness and Safety (BEST) Initiative using a rapid cycle analysis or near real-time monitoring framework which enables early detection of potential safety signals. This near real-time surveillance effort is part of the US government’s larger ongoing post-authorization monitoring and real-world evidence to ensure the safety of COVID-19 vaccines in children.
Methods
Data Sources
This study used administrative claims data from Optum, HealthCore (HealthCore Integrated Research Database), and CVS Health (Aetna Enterprise Data Warehouse) containing longitudinal medical and pharmacy claims data supplemented with vaccination data from participating local and state Immunization Information Systems (IIS) (Supplemental Table 1).4
Study Population
We included health plan members aged 5-17 years who received BNT162b2 vaccination from the earliest date of emergency use authorization by age group through 6/25/2022 (Optum), 5/6/2022 (HealthCore), and 5/31/2022 (CVS Health). The study population was divided into three age groups according to the vaccine authorization schedule (5-11, 12-15, and 16-17 years). We required continuous enrollment in a medical health insurance plan from the start of an outcome-specific clean window to the COVID-19 vaccination.
Exposure and Follow Up
Exposure was defined as the receipt of BNT162b2 COVID-19 vaccine (Supplemental Table 2). Dose number was assigned chronologically. The primary analyses included primary series doses (Dose 1 + Dose 2). Secondary analyses included stratification by Dose 1, 2, 3/booster. Follow up began on the date of eligible vaccination and was censored at subsequent vaccination, death, disenrollment, end of risk window, or end of study period.
Health Outcomes
Using claims-based algorithms, we monitored 20 health outcomes selected through clinical consultation and literature review. Among them, 13 were assessed using sequential testing and seven were monitored descriptively due to lack of historical comparator.5 Myocarditis/pericarditis was assessed with varying risk windows and care settings based on evidence from prior surveillance efforts and clinician input (Supplemental Table 3).
Sequential Testing
We conducted monthly sequential testing using the Poisson Maximized Sequential Probability Ratio Test6 and generated incidence rate ratios (RR) of observed outcome rates compared to database-specific historical (expected) rates. Pre-COVID-19-vaccination historical rates were adjusted for claims processing delay and standardized by age and sex where case counts permitted.7 One-tailed tests were used with a null hypothesis that the observed rate was no greater than the historical comparator rate beyond a pre-specified test margin with an overall alpha of 1%. A statistical signal occurred if the log likelihood ratio exceeded a critical value. Surveillance continued until a signal was detected or the prespecified maximum surveillance length was reached. 5
Medical Record Review
Medical record review was conducted for myocarditis/pericarditis to assess the validity of cases identified by claims-based algorithms. Clinical adjudicators used the Brighton collaboration definition to classify cases, and records meeting the confirmed or probable Brighton classifications were considered a true case.8
Results
Descriptive Monitoring
Approximately 5.9 million BNT162b2 COVID-19 vaccine doses were administered to over 3 million commercial health plan enrollees aged 5-17 years across the three databases; 1,999,550 doses were observed in 1,000,895 enrollees in CVS Health; 2,033,212 doses in 1,078,712 enrollees in HealthCore; and 1,869,063 doses in 937,745 enrollees in Optum database (Figure 1). Demographic characteristics of the vaccinated populations were largely similar across databases (Table 1). We observed low incidence for the seven outcomes monitored descriptively in all three databases (<25 events of each outcome in each database; Supplemental Table 5).
Sequential Testing
Of the 13 outcomes sequentially tested, only myocarditis/pericarditis met the threshold for a statistical signal in any of the three databases. In the primary series analyses, myocarditis/pericarditis signaled in all databases for ages 12-15 and 16-17 years following BNT162b2 COVID-19 vaccination in all definitions of the myocarditis/pericarditis outcome (Table 2). In the dose-specific analyses, a signal was detected following Dose 2 in all definitions of the myocarditis/pericarditis outcome in ages 12-15 and 16-17 years in all data partners.
Following Dose 3, signals were detected for some of the definitions of the myocarditis/pericarditis outcome in ages 16-17 years in CVS Health and ages 12-15 years in HealthCore databases (Supplemental Table 4). No signals were observed in ages 5-11.
Medical Record Review
Of the 153 identified cases of myocarditis/pericarditis aged 12-17, medical record review was conducted for a sample of 37 cases whose charts were obtainable. Twenty-seven cases (73.0%) were confirmed as true myocarditis/pericarditis cases, of which 23 were male (93%) and 19 (70.4%) were hospitalized with a mean length of hospital stay of 2.8 days (median = 2 days). Themean time from vaccination to presentation for care for myocarditis/pericarditis was 6.8 days (median = 3 days).
Discussion
Our near real-time monitoring results of 20 pre-specified health outcomes in the pediatric population provide reassuring real-world evidence of the safety of the BNT162b2 COVID-19 vaccine in children and adolescents. The signal detected for myocarditis/pericarditis is consistent with published peer-reviewed publications demonstrating an elevated risk of myocarditis/pericarditis following mRNA vaccines, especially among younger males aged 12-29 years.9-11 It should be noted that myocarditis/pericarditis is a rare event, with an average incidence of 39.4 cases per million doses administered in ages 5-17 years within 7 days after BNT162b2 COVID-19 vaccination.12-13 We did not detect a signal for myocarditis/pericarditis in young children aged 5-11 years, which is consistent with reports from other surveillance systems.14-15
This study has several strengths. First, the study included a large, geographically diverse population from three U.S. commercial health insurance databases. Due to availability of more complete information from claims supplemented with IIS data and a short data lag from health encounters, we were able to monitor BNT162b2 COVID-19 vaccine safety in a near real-time manner. Additionally, identified myocarditis/pericarditis cases were validated through medical record review.
The study also has some limitations. First, this study only covers data on a commercially insured pediatric population and may not be nationally representative. We used a rapid monitoring method designed for early detection of potentially increased risk of health outcomes with limited confounding adjustment. Therefore, results of this study do not establish a causal relationship between the vaccine and health outcomes; signals should be further evaluated. Furthermore, the study may have limited power to detect small increases in risk of outcomes in certain subgroups with more recent authorizations such as booster doses in younger children. Analyses were not stratified by sex, which is an important demographic characteristic for certain outcomes.
FDA continues to monitor the safety of COVID-19 vaccines in the pediatric population and has expanded the framework to include additional age groups and vaccine brands with updated authorizations. The FDA BEST Initiative plays a major role in the larger U.S. federal government vaccine safety monitoring efforts and further supports decision-making concerning the safety of COVID-19 vaccines by healthcare professionals and the American public.
Data Availability
The study protocol was publicly posted as referenced in the manuscript before data analyses and related documents can be made available where needed, by contacting the corresponding author. De-identified participant data will not be shared without approval from the data partners.
Funder
The US Food and Drug Administration provided funding for this study and contributed as follows: led the design of the study, interpretation of the results, writing of the manuscript, decision to submit, and made contributions to the coordination of data collection and analysis of the data.
Disclosures
Annemarie Kline and Cheryl McMahill-Walraven worked on grants, subcontracts, or contracts from Harvard Pilgrim Health Care Institute, Brown University (National Institute on Aging/IMPACT Collaboratory), Reagan Udall Foundation for the FDA, Academy of Managed Care pharmacy’s Biologics and Biosimilar Collective Intelligence Consortium (BBCIC), TherapeuticsMD, Reachnet (Louisiana Public Health Institute), IQVIA, Pfizer, Healthcore, and Patient Centered Outcomes Research Institute as employees of CVS Health and report stock or stock options from CVS Health; Danielle Cooper worked on grants, subcontracts, or contracts from Harvard Pilgrim Health Care Institute, Brown University (National Institute on Aging/IMPACT Collaboratory), Reagan Udall Foundation for the FDA, Academy of Managed Care pharmacy’s Biologics and Biosimilar Collective Intelligence Consortium (BBCIC), TherapeuticsMD, Reachnet (Louisiana Public Health Institute), IQVIA, Pfizer, Healthcore, and Westat as an employee of CVS Health and reports stock or stock option from CVS Health; Charlalynn Harris received grants or contracts from Harvard Pilgrim Health Care Institute, Reagan Udall Foundation for the FDA, and Academy of Managed Care pharmacy’s Biologics and Biosimilar Collective Intelligence Consortium (BBCIC) as an employee of CVS Health and reports stock or stock options in CVS Health; Robin Clifford and John Seeger report stock or stock options in UnitedHealth Group; Daniel Beachler is an employee of HealthCore, Inc. who has previously contracted with Pfizer Inc. for separate projects. No other authors report relevant disclosures.
Additional Contributions
We thank Tainya C. Clarke and Kristin A. Sepúlveda of the US Food and Drug Administration; Anchi Lo, Bing Lyu, Derek Phan, Gyanada Acharya, Laurie Feinberg, Lloyd Marks, Minisha Kochar, Sandia Akhtar, Shruti Parulekar, Stella Zhu, William (Trey) Minter, Yeerae Kim, Yixin Jiao, and Zhiruo (Cassie) Wan of Acumen; Carla Brennan of CVS Health; Shiva Vojjala, Ramya Avula, Shiva Chaudhary, Shanthi P Sagare, Ramin Riahi, Navyatha Namburu, and Grace Stockbower of HealthCore; Michael Goodman, Michael Bruhn, and Ruth Weed of IQVIA; Grace Yang, Karen Schneider, Rebecca Braun, Megan Ketchell, and Kate Federici of Optum.