Abstract
Background The CDC recommends serial rapid antigen assay collection within congregate facilities for screening and outbreak testing. Though modeling and observational studies from community and long-term care facilities have shown serial collection provides adequate sensitivity and specificity, the diagnostic accuracy of this testing strategy within correctional facilities remains unknown.
Methods Using Connecticut Department of Corrections (DOC) data from November 21st 2020 to June 15th 2021, we estimated the accuracy of a rapid assay, BinaxNOW, under three collection strategies, a single test in isolation and two and three serial tests separated by 1-4 day intervals. Diagnostic accuracy metrics were estimated in relation to RT-PCRs collected within one day before the first or after the last included rapid antigen tests in a series.
Results Of the 17,669 residents who contributed at least one RT-PCR or rapid antigen during the study period, 3,979 contributed ≥1 paired rapid antigen test series. In relation to RT-PCR, the three-rapid antigen test strategy had a sensitivity of 89.6% (95% confidence intervals: 86.1-92.6%) and specificity of 97.2% (CI: 95.1-98.3%). The sensitivities for two and one-rapid antigen test strategy were 75.2% and 52.8%, respectively, and the specificities were 98.5% and 99.4%, respectively. The sensitivity was higher among symptomatic residents and when the RT-PCR was collected before the rapid antigen tests.
Conclusions We found the serial collection of an antigen test resulted in high diagnostic accuracy. These findings support serial testing within correctional facilities for outbreak investigation, screening, and when rapid detection is required (such as intakes or transfers).
Introduction
Within the United States, state and federal run correctional facilities have experienced high transmission rates of SARS-CoV-2 and remain high-risk settings for COVID-19.[1,2] In fact, data from September through November 2020 show that residents of Federal Bureau of Prisons were 4.7 times more likely to become infected with SARS-CoV-2 and 2.6 times more likely to die from COVID-19 than general US residents.[2] Despite the development of COVID-19 vaccines and vaccination programs for incarcerated populations, vaccine coverage remains below that needed for population level protection.[3–10] Rapid and accurate SAR-CoV-2 testing will therefore remain a key component of infection prevention within correctional facilities.
In August 2020, the Food and Drug Administration (FDA) issued an emergency use authorization of Abbott’s BinaxNOW, a COVID-19 rapid antigen test.[11,12] Compared with reverse transcription polymerase chain reaction (RT-PCR), the rapid turnaround time and low cost of rapid antigen tests makes them a cost-effective strategy for congregate settings, where transmission risk is high and implementation of serial mass screening is feasible.[13] Unfortunately, prior studies from community, educational, and long-term care facility settings found the sensitivity of rapid antigen tests to be poor to moderate (53-77%) and to be lower among asymptomatic individuals and individuals early or late in their course of infection.[14–19] These findings, thus, call into question the use of rapid antigen tests as single point of care tests.
Single test collection is not, however, the intended testing strategy for rapid antigen tests outside of healthcare settings.[12,20,21] Instead, both the manufactures and the FDA advice serial collection of at least two tests.[12,20] In alignment with the intended use, the Centers for Disease Control (CDC) recommends serial collection of rapid antigen testing within congregate settings for screening and during outbreaks regardless of symptom presentation.[22] This guidance is supported by modeled evidence that outbreak control depends largely on frequency and speed of testing and observational data showing that serial testing improves the sensitivity of rapid antigen tests in both nursing home and community settings.[13,15,23,24] However, the value of serial testing within correctional facilities remains unknown.
Herein, we present findings of a study that evaluated the accuracy of serial BinaxNOW rapid antigen testing during a mass screening and testing program implemented by the Connecticut Department of Corrections (DOC) during the COVID-19 pandemic wave in late 2020 and early 2021. We specifically compared the accuracy of the rapid antigen test relative to RT-PCR under three collection strategies: serial testing of up to three negative rapid antigen tests, serial testing of up to two negative rapid antigen tests, and rapid antigen collection in isolation.
Methods
Setting and Specimen Collection
On November 21, 2020, the DOC initiated the implementation of rapid antigen test (BinaxNOW) collection into their SARS-CoV-2 testing program for symptomatic testing, contact tracing, testing of residents undergoing admission to the correctional facilities and inter-facility transfer and mass voluntary asymptomatic screening. For each instance of rapid antigen use, the DOC guidelines recommend the serial collection of up to three negative rapid antigen tests taken on day one, four, and seven. Due to concerns around the sensitivity of the rapid antigen test, phased implementation of rapid antigen testing with confirmatory RT-PCR was performed. While undergoing serial test collection, residents of DOC facilities were placed in quarantine or isolation.
Trained medical staff collected anterior nasal and nasopharyngeal swab specimens for rapid antigen and RT-PCR testing. Quest Diagnostic facilities performed SARS-CoV-2 RT-PCR testing of nasopharnyngeal swab specimens and defined positive tests as having a cycle threshold value less than 40.[25] At the time rapid antigen test collection was implemented, the DOC oversaw 17 facilities with a resident census of around 10,000 residents (9,945 residents on July 1st, 2020).[26,27] The Yale University Institutional Review Board classified this study as public health surveillance.
Data Collection and Cohort Development
Using resident and testing data queried from internally maintained DOC databases, we retrospectively identified all rapid antigen (BinaxNOW) and RT-PCR testing records from November 21st, 2020 to June 15th, 2021. Following the removal of duplicated records (same day, assay, and assay results) and reporting errors (negative recorded on the same day as a positive), we identified rapid antigen test series (rapid antigen tests collected within one and four days of each other or tests collected in the absence of any test in the prior or following four days). We then retained the first three test of each series, resulting in series of between one and three rapid antigen tests (eFigure1).
We paired the rapid antigen test series with RT-PCR tests collected between one day prior to the first and one day following the last rapid antigen test of the series (eFigure1). In the event of multiple RT-PCR matches per series, ordered preference was given to positive RT-PCRs, RT-PCRs collected on the same day or between rapid antigen tests of a series, and RT-PCRs collected before the rapid antigen test series. We defined symptoms as the presence or absence of COVID-19 related symptoms reported at the time of the rapid antigen test.[28]
Statistical Approach: Diagnostic Accuracy
The characteristics of residents with at least one RT-PCR paired rapid antigen test series were summarized by presence of RT-PCR positive SARS-CoV-2 events using counts and percentages for categorical factors and means and standard errors for continuous factors. We determined the diagnostic accuracy of rapid antigen test in relation to RT-PCR using sensitivity, specificity, and positive and negative predictive value (PPV/NPV). We estimated the sensitivity and specificity of the first, second, and third test of all series using generalized estimating equations (GEE) with robust standard errors and a logit link (eFigure2). With these estimates, we calculated the sensitivity and specificity for each testing strategy using the following equations[29]:
Where Test1sen/spec was the sensitivity or specificity of the first rapid antigen test of all series, Test2sen/spec was the sensitivity or specificity of the second rapid antigen test of all series of two or more rapid antigen tests, and Test3sen/spec was the sensitivity or specificity of the third rapid antigen test of all series of three rapid antigen tests (eFigure2). We propagated the uncertainty through the serial testing equations using posterior simulation of 1000 random draws of the GEE estimate.[30,31] Additionally, we estimated the diagnostic accuracy stratified by the following a priori selected factors: age, symptom presence, and test order. We tested for additive differences in the diagnostic accuracy of the stratified samples by subtracting the draws of each sample from a selected reference category.
For the PPV and NPV, we simulated 1000 average daily prevalence estimates using a Poisson regression with an outcome of positive SARS-CoV-2 test (either rapid antigen or RT-PCR test). To reduce the risk of including multiple positive tests from the same testing event, we excluded positive events within 5 days of each other. With the estimated prevalence, sensitivity, and specificity, we estimated the PPV and NPV at the draw level.[32] For each accuracy metric, we calculated 95% confidence intervals as the 2.5 and 97.5 percentiles.[31] All analyses were conducted in R 4.1.0 using the geepack and multcomp packages.[33,34]
Sensitivity Analyses
Because of the retrospective nature of the presented analysis, we had to make assumptions around test selection and the appropriate timing of RT-PCR and rapid antigen testing among included pairs. To test the robustness of our findings, we performed sensitivity analyses where we selected first, second and third series tests at random instead of the observed order, limited to rapid antigen test series where tests were collected exactly three days apart, and invoked different selection approaches in the event of multiple RT-PCRs linked to a rapid antigen test series. Unless otherwise stated, all sensitivity analyses were performed among the full sample and mirrored the primary analysis for all but their stated variation.
Results
Between November 21st 2020 and June 15th 2021, 128,986 RT-PCR and 30,112 rapid antigen tests were collected among 17,669 DOC residents (Figure1). Of the 3,367 residents who contributed at least one RT-PCR paired rapid antigen test series, 583 (17.3%) contributed at least one rapid antigen test series paired with a positive RT-PCR. The majority of residents were male (74.9%), and the most frequently observed race was Black (40.7%). Individuals who experienced a paired RT-PCR positive SARS-CoV-2 event were demographically similar to individuals who did not experience a paired RT-PCR positive SARS-CoV-2 event.
SARS-CoV-2 daily testing volumes from Connecticut run correctional facilities by date for A. all collected paired and unpaired tests, B. RT-PCR paired rapid antigen test series, C. all positive RT-PCR and rapid antigen tests, and D. positive RT-PCR paired rapid antigen test series. Red, Grey, Yellow, Green, Blue lines show volumes of SARS-CoV-2 RT-PCR only, rapid antigen test only, paired RT-PCR series of one rapid antigen test, paired RT-PCR series of two rapid antigen tests, paired RT-PCR series of three rapid antigen tests, respectively. Series were defined as rapid antigen tests collected within one and four days of each other or tests collected in the absence of any test in the prior or following four days. Paired series were defined as rapid antigen test series with RT-PCR collected between one day prior to the first and one day following the last test of the series.
Residents contributed an average of 1.2 RT-PCR paired rapid antigen test series (3,884 pairs/3,367 residents) and six people contributed more than one positive RT-PCR paired rapid antigen test series (Table1). Among the 3,884 included series, 692 consisted of at least two rapid antigen tests and 358 consisted of three rapid antigen tests (Figure 2). The second rapid antigen test of included series were more likely to be collected prior to the paired RT-PCR (62.1%) than the first (37.3%) and third (23.7%) test of included series. The proportion of tests collected among symptomatic and asymptomatic individuals was similar between the first, second, and third tests of included series (eTable1).
Demographic Characteristics of Residents of Connecticut State Correctional Facilities with Time Matched BinaxNOW and RT-PCR tests by Occurrence of SARS-CoV-2 Infection
Flowchart of residents of Connecticut run correctional facilities and rapid antigen test series included in our analysis. Series were defined as rapid antigen tests collected within one and four days of each other or tests collected in the absence of any test in the prior or following four days. Paired series were defined as rapid antigen test series with RT-PCR collected between one day prior to the first and one day following the last test of the series.
Most rapid antigen test of included series were negative (first test: 91.6%, second test: 86.7%, third test: 88.5%). Of the negative rapid antigen tests, most were paired with negative RT-PCRs (first test: 84.3%; second test: 73.8%; third test: 80.4%; eTable2). Relative to RT-PCR, the sensitivity of the first, second and third-rapid antigen tests of included series were 51.6% (95% confidence interval: 47.6-55.6%), 48.6% (CI: 41.2-56.0%) and 58.0% (CI: 46.2-69.2%), respectively. This resulted in sensitivities of 75.2% (CI: 71.0-79.2%) and 89.6% (86.1-92.6%) for the serial collection of two and three rapid antigen tests, respectively. The specificity of each rapid antigen test of included series was above 98%. The specificity of two and three serially collected rapid antigen tests was 97.8% (96.4-98.6%) and 97.2% (95.1-98.4%), respectively (Table2). The PPV, based on an observed prevalence of 57 (CI: 53-62) cases per 100,000 residents, was highest for the one-rapid antigen test strategy (63.5%) and lowest for the three-rapid antigen test strategies (50.1%). The NPV for each rapid antigen test collection strategy was high (one-rapid antigen test: 98.6%, two-rapid antigen tests: 99.2%, three-rapid antigen tests: 99.7%, eTable3).
Rapid Antigen Test Accuracy Relative to RT-PCR Among Residents of Connecticut State Correctional Facilities Under Varying Collection Strategies
The sensitivity for each collection strategy was significantly higher among symptomatic residents (one-rapid antigen test: 62.9%, two-rapid antigen tests: 82.2%, three-rapid antigen tests: 95.6%) than among asymptomatic patients (one-rapid antigen test: 47.4%, two-rapid antigen tests: 72.0%, three-rapid antigen tests: 85.5%). The sensitivity was higher when the rapid antigen test was collected on the same day as the RT-PCR (one-rapid antigen test: 73.5%; two-rapid antigen tests: 93.6%, three-rapid antigen tests: no sample) than when the rapid antigen test was collected prior (one-rapid antigen test: 15.2%; two-rapid antigen tests: 26.9%, three-rapid antigen tests: 39.4%). The sensitivity of the three-rapid antigen test collection strategy was significantly lower for individuals more than 37 years of age than individuals 37 or fewer years of age (difference: 8.3 CI: 1.8-14.6; Table3).
Rapid Antigen Test Accuracy Relative to RT-PCR Among Residents of Connecticut State Correctional Facilities Under Varying Collection Strategies by Demographic and Clinical Characteristics
The specificity for each rapid antigen test collection strategy was significantly lower among symptomatic residents (one-rapid antigen test: 94.7%, two-rapid antigen tests: 85.1%, three-rapid antigen tests: 84.3%) than asymptomatic residents (one-rapid antigen test: 99.2%, two-rapid antigen tests: 98.1%, three-rapid antigen tests: 97.5%). The specificities did not vary significantly by age or timing of tests (Table3).
Sensitivity Analyses
We performed three sets of sensitivity analyses to test the robustness of our findings to varying rapid antigen test collection orders, rapid antigen test collection windows, and RT-PCR pair selection. The sensitivity of the one-rapid antigen test, two-rapid antigen test, and three-rapid antigen test strategies ranged from 51.0-75.4%, 76.3-86.1%, 87.2-100%, respectively. The highest sensitivities were observed when we restricted to rapid antigen test series with tests collected exactly three days apart (eTable4). The lowest sensitivities were observed when we randomly selected the first, second, and third test of each series (eTable5). As with the primary analysis, we observed high specificities for each testing strategy (one-rapid antigen test: 99.0-99.6%, two-rapid antigen test: 97.7-98.9%, three-rapid antigen test: 96.9-98.9%; eTables4-5).
Discussion
We evaluated the predictive accuracy of the rapid COVID-19 antigen test, BinaxNOW, in relation to RT-PCR under three different collection strategies (a single test in isolation and two and three serial tests separated by 1-4 day intervals) among residents of Connecticut state prisons and jails between November 21st, 2020 and June 15th, 2021. The three test strategy is currently recommended by Connecticut DOC.[35] In alignment with diagnostic accuracy estimates from other congregate settings, we found that rapid antigen tests had a low sensitivity when collected in isolation, but the sensitivity increased significantly when rapid antigen tests were collected in pairs and triplets.[14,15,23] Further, we found that the sensitivity for all rapid antigen test collection strategies was significantly higher among symptomatic individuals than asymptomatic individuals.
In relation to RT-PCR, we found the current DOC rapid antigen test collection strategy, three-rapid antigen test, had a high sensitivity (90%) and specificity (97%). While the specificity of the two and one-rapid antigen test collection strategies were higher (98% and 99%, respectively), the sensitivity for these less intensive collection strategies were significantly lower (75% and 53%, respectively). These findings suggest that, among 100 residents infected with SARS-CoV-2, 90 would be captured by the three-rapid antigen test strategy. Compared with this strategy, the two-rapid antigen test and the one-rapid antigen test strategies would miss an additional 15 and 37 infected individuals, respectively. Conversely, the three-rapid antigen test strategy would only misdiagnose three out of 100 uninfected residents. Though this is two more than the one-rapid antigen test strategy, the cost of false negatives (missed infections) far outweighs the cost of false positives (excess isolation) under scenarios of highly transmissible infectious diseases, such as SARS-CoV-2.[36]
In alignment with prior studies, we found the sensitivity of rapid antigen testing to be significantly higher later in the course of infection (rapid antigen tests collected on the same day or after RT-PCR) and among residents exhibiting COVID-19 symptoms.[37,38] However, contrary to previous findings, we found the specificity to be significantly lower among symptomatic residents than among asymptomatic residents.[38–40] While, the magnitude of this difference was minimal for the one-rapid antigen test strategy (specificity among symptomatic residents was 5% lower than asymptomatic residents), the difference was larger (13%) for the three-rapid antigen test strategy. Despite this meaningful difference in the specificity among symptomatic and asymptomatic residents under the three-rapid antigen test strategy, the specificity among symptomatic residents remained high (84%). Thus, this difference in specificity does not invalidate the use of the three-test strategy for symptomatic residents of correctional facilities.
Taken collectively, our findings support the use of serial rapid antigen testing under scenarios when PCR turnround time is long, rapid detection is required, or when isolating/quarantining all exposed individuals is unfeasible. Such scenarios include times of unknown exposure (intake and transfers), contact tracing, asymptomatic and symptomatic screening, and during outbreak investigations. These recommendations stem from the rapid turnaround time and low cost of a single test, the collectively high diagnostic accuracy of serial collection, and the ability of serial collection strategies to detect events under continuous exposure scenarios. Though our findings speak predominately to the value of serial collection resulting from a single exposure, serial collection provides additional benefit through capture of infections from exposures that occurred immediately prior to or following the first collected test.[13] The combination of these benefits, thus, may results in more rapid isolation of infected individuals, and in turn, a reduction in facility wide transmission.
Limitations and Strengths
Our study was subject to limitations typical of retrospective diagnostic validation analyses. First, race/ethnicity was missing for a large portion of the population and we were unable to test for differences in test accuracy by race. However, it is unlikely that race would impact the diagnostic accuracy of the rapid antigen test. Second, we relied on a reference outcome of RT-PCR positivity, which is an imperfect indicator of infection. In addition, in the absence of cycle threshold values, we were unable to tests the impact of viral load on rapid antigen test performance. Third, our accuracy estimates relied on collected test that may be biased towards department specific testing practices. However, we observed similar results when we selected the first, second, and third test of each series at random (eTable5). Finally, our study was conducted prior to the large Delta and Omicron waves of 2021. While the diagnostic accuracy of rapid antigen tests likely varies between the different variants, we believe the benefits of serial collection will hold.
Our study had several strengths including our large sample of paired assays collected among a diverse population of Connecticut State correctional facility residents. This large sample allowed us to estimate and compare the diagnostic accuracy of three different collection strategies, including the three-test strategy employed by the Connecticut DOC. Our large sample also allowed us to examine characteristics associated with the accuracy of the rapid antigen test within correctional facilities settings and speak to the use of different collection strategies based on these characteristics. Additionally, through the inclusion of numerous sensitivity analyses, we were able to show that our findings were not the result of the data cleaning or modeling assumptions we employed within this analysis. Finally, we were able to include all unique rapid antigen test sets and account for within person correlation in our uncertainty intervals using generalized estimating equations.
Conclusion
Compared with singularly collected tests, we found that serial collection of BinaxNOW rapid antigen tests resulted in meaningfully higher sensitivities and comparably high specificities among residents in state correctional facilities. We found this held for both asymptomatic and symptomatic residents and regardless of rapid antigen test collection time relative to RT-PCR collection. These findings speak to the utility of serially collected rapid antigen tests within correctional facilities for asymptomatic and symptomatic screening, contact tracing, and during outbreak investigations. If employed under such scenarios, rapid antigen testing may result in faster isolation of infected individuals and reduce transmission within facilities.
Data Availability
Data Use Agreement: The data used in this study belongs to the Connecticut Department of Corrections. Qualified researchers may submit a data share request for de-identified patient level data by contacting the corresponding author with a detailed description of the research question.
Additional Contributions
We thank the Connecticut Department of Corrections and the healthcare workers in each correctional facility for administrative support and administering RT-PCR and rapid antigen tests. None of the individuals received compensation for their contribution.
Data Use Agreement
The data used in this study belongs to the Connecticut Department of Corrections. Qualified researchers may submit a data share request for de-identified patient level data by contacting the corresponding author with a detailed description of the research question.
Funding
Connecticut Department of Public Health Emerging Infections Program (EIP): COVID-19 contract (DPH log # 2021-0071-3)
Author Contributions
Margaret L. Lind had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Ko, Kennedy, Richeson
Acquisition, analysis, or interpretation of data: Schultes, Lind
Drafting of the manuscript: Lind, Schultes, Robertson, Ko, Cummings
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Lind
Administrative, technical, or material support:
Supervision: Ko, Kennedy, Richeson, Cummings
Conflict of Interest Disclosures
A.I.K serves as an expert panel member for Reckitt Global Hygiene Institute, scientific advisory board member for Revelar Biotherapeutics and a consultant for Tata Medical and Diagnostics and Regeneron Pharmaceuticals, and has received grants from Merck, Regeneron Pharmaceuticals and Tata Medical and Diagnostics for research related to COVID-19, all of which are outside the scope of the submitted work. Other authors declare no conflict of interest.