Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Performance of Rapid Antigen Tests to Detect Symptomatic and Asymptomatic SARS-CoV-2 Infection

View ORCID ProfileApurv Soni, View ORCID ProfileCarly Herbert, View ORCID ProfileHonghuang Lin, Yi Yan, Caitlin Pretz, Pamela Stamegna, Biqi Wang, View ORCID ProfileTaylor Orwig, View ORCID ProfileColton Wright, Seanan Tarrant, View ORCID ProfileStephanie Behar, View ORCID ProfileThejas Suvarna, View ORCID ProfileSummer Schrader, View ORCID ProfileEmma Harman, View ORCID ProfileChris Nowak, View ORCID ProfileVik Kheterpal, View ORCID ProfileLokinendi V Rao, Lisa Cashman, View ORCID ProfileElizabeth Orvek, View ORCID ProfileDidem Ayturk, Laura Gibson, Adrian Zai, Steven Wong, Peter Lazar, View ORCID ProfileZiyue Wang, Andreas Filippaios, View ORCID ProfileBruce Barton, View ORCID ProfileChad J. Achenbach, View ORCID ProfileRobert L. Murphy, View ORCID ProfileMatthew Robinson, View ORCID ProfileYukari C. Manabe, Shishir Pandey, View ORCID ProfileAndres Colubri, View ORCID ProfileLaurel O’Connor, View ORCID ProfileStephenie C. Lemon, View ORCID ProfileNisha Fahey, View ORCID ProfileKatherine L Luzuriaga, View ORCID ProfileNathaniel Hafer, Kristian Roth, Toby Lowe, Timothy Stenzel, View ORCID ProfileWilliam Heetderks, View ORCID ProfileJohn Broach, View ORCID ProfileDavid D McManus
doi: https://doi.org/10.1101/2022.08.05.22278466
Apurv Soni
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
2Division of Health System Science, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
3Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, Worcester, MA, USA
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Apurv Soni
  • For correspondence: apurv.soni@umassmed.edu
Carly Herbert
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
BA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Carly Herbert
Honghuang Lin
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
2Division of Health System Science, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Honghuang Lin
Yi Yan
4OHT7 Office of Product Evaluation and Quality, Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Caitlin Pretz
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
MS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Pamela Stamegna
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Biqi Wang
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
2Division of Health System Science, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Taylor Orwig
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
BS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Taylor Orwig
Colton Wright
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
MS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Colton Wright
Seanan Tarrant
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
BS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Stephanie Behar
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
BA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Stephanie Behar
Thejas Suvarna
5CareEvolution, LLC, Ann Arbor, MI, USA
BBA, BS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Thejas Suvarna
Summer Schrader
5CareEvolution, LLC, Ann Arbor, MI, USA
BA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Summer Schrader
Emma Harman
5CareEvolution, LLC, Ann Arbor, MI, USA
MPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Emma Harman
Chris Nowak
5CareEvolution, LLC, Ann Arbor, MI, USA
BA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Chris Nowak
Vik Kheterpal
5CareEvolution, LLC, Ann Arbor, MI, USA
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Vik Kheterpal
Lokinendi V Rao
6Quest Diagnostics, Marlborough, MA, USA
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Lokinendi V Rao
Lisa Cashman
6Quest Diagnostics, Marlborough, MA, USA
MPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Elizabeth Orvek
3Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, Worcester, MA, USA
MS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Elizabeth Orvek
Didem Ayturk
3Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, Worcester, MA, USA
MS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Didem Ayturk
Laura Gibson
7Division of Infectious Disease, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Adrian Zai
3Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, Worcester, MA, USA
MD PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Steven Wong
3Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, Worcester, MA, USA
BA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Peter Lazar
3Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, Worcester, MA, USA
BS
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ziyue Wang
8Division of Cardiology, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
MPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Ziyue Wang
Andreas Filippaios
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Bruce Barton
3Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, Worcester, MA, USA
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Bruce Barton
Chad J. Achenbach
9Division of Infectious Disease, Department of Medicine, Havey Institute for Global Health, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
MD, MPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Chad J. Achenbach
Robert L. Murphy
9Division of Infectious Disease, Department of Medicine, Havey Institute for Global Health, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Robert L. Murphy
Matthew Robinson
10Division of Infectious Disease, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Matthew Robinson
Yukari C. Manabe
10Division of Infectious Disease, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Yukari C. Manabe
Shishir Pandey
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
BE
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andres Colubri
11Department of Microbiology and Physiological Systems, University of Massachusetts Chan Medical School, Worcester, MA, USA
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Andres Colubri
Laurel O’Connor
12Department of Emergency Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Laurel O’Connor
Stephenie C. Lemon
3Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, Worcester, MA, USA
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Stephenie C. Lemon
Nisha Fahey
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
3Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, Worcester, MA, USA
13Department of Pediatrics, University of Massachusetts Chan Medical School, Worcester, MA, USA
DO ScM
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Nisha Fahey
Katherine L Luzuriaga
14University of Massachusetts Center for Clinical and Translational Science, University of Massachusetts Chan Medical School, Worcester, MA, USA
15Program in Molecular Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Katherine L Luzuriaga
Nathaniel Hafer
14University of Massachusetts Center for Clinical and Translational Science, University of Massachusetts Chan Medical School, Worcester, MA, USA
15Program in Molecular Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Nathaniel Hafer
Kristian Roth
4OHT7 Office of Product Evaluation and Quality, Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Toby Lowe
4OHT7 Office of Product Evaluation and Quality, Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Timothy Stenzel
16Division of Microbiology, OHT7 Office of Product Evaluation and Quality, Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland
MD PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
William Heetderks
17National Institute of Biomedical Imaging and Bioengineering, NIH, via contract with Kelly Services, Bethesda, MD, USA
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for William Heetderks
John Broach
12Department of Emergency Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
MD, MPH, MBA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for John Broach
David D McManus
1Program in Digital Medicine, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
2Division of Health System Science, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
8Division of Cardiology, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, USA
MD, ScM
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for David D McManus
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

Abstract

Background Performance of rapid antigen tests for SARS-CoV-2 (Ag-RDT) varies over the course of an infection, and their performance in screening for SARS-CoV-2 is not well established. We aimed to evaluate performance of Ag-RDT for detection of SARS-CoV-2 for symptomatic and asymptomatic participants.

Methods Participants >2 years old across the United States enrolled in the study between October 2021 and February 2022. Participants completed Ag-RDT and molecular testing (RT-PCR) for SARS-CoV-2 every 48 hours for 15 days. This analysis was limited to participants who were asymptomatic and tested negative on their first day of study participation. Onset of infection was defined as the day of first positive RT-PCR result. Sensitivity of Ag-RDT was measured based on testing once, twice (after 48-hours), and thrice (after 96 hours). Analysis was repeated for different Days Post Index PCR Positivity (DPIPP) and stratified based on symptom-status.

Results In total, 5,609 of 7,361 participants were eligible for this analysis. Among 154 participants who tested positive for SARS-CoV-2, 97 were asymptomatic and 57 had symptoms at infection onset. Serial testing with Ag-RDT twice 48-hours apart resulted in an aggregated sensitivity of 93.4% (95% CI: 89.1-96.1%) among symptomatic participants on DPIPP 0-6. Excluding singleton positives, aggregated sensitivity on DPIPP 0-6 for two-time serial-testing among asymptomatic participants was lower at 62.7% (54.7-70.0%) but improved to 79.0% (71.0-85.3%) with testing three times at 48-hour intervals.

Discussion Performance of Ag-RDT was optimized when asymptomatic participants tested three-times at 48-hour intervals and when symptomatic participants tested two-times separated by 48-hours.

Introduction

SARS-CoV-2 diagnostic testing remains a cornerstone in our nation’s fight against COVID-19, and at-home rapid antigen tests (Ag-RDTs), while not perfect, provide a fast and convenient testing option. This type of test is available without a prescription (i.e., over-the-counter [OTC]), easy-to-use, widely available, and, in some cases, preferred by the population over molecular assays that require appointments, waiting in-line at testing centers, and waiting 24-48 hours for results.1–3 Despite their popularity, key gaps remain in our understanding of these tests, notably their performance as a screening tool among asymptomatic people. Reports on Ag-RDT performance among individuals testing while they are asymptomatic have been highly varied, ranging from sensitivities of 35.8% to 71% in cross-sectional screening evaluations.4,5 However, performance has typically been evaluated based on the single use of Ag-RDTs, and few studies have evaluated serial testing performance of Ag-RDTs among asymptomatic individuals.

Furthermore, United States Food and Drug Administration (FDA) emergency use authorization (EUA) of OTC antigen tests required a post-authorization demonstration of Ag-RDT performance in a population with asymptomatic infection using serial testing. This manuscript describes primary findings from a large study designed in coordination with the National Institutes of Health (NIH), Food and Drug Administration (FDA), and three major rapid antigen test manufacturers to evaluate the performance of serial testing using rapid antigen tests for detection of SARS-CoV-2 among asymptomatic individuals within the first week of infection. A primary goal of this study was to provide broadly applicable data that could be leveraged to satisfy the post-authorization requirement for all authorized OTC antigen tests.

Methods

Study Population and Design

Between October 18, 2021 and January 31, 2022, this 15-day prospective cohort study enrolled participants over the age of two years from across the country through a novel digital site-less study protocol. Details of the study design and protocol are described elsewhere.6 This study was approved by WIRB-Copernicus Group (WCG) Institutional Review Board (20214875). In brief, participants were eligible to enroll through a smartphone app if they did not have a SARS-CoV-2 infection in the prior three months, were without any symptoms in the 14 days prior to enrollment and were able to drop-off prepaid envelopes with nasal swab samples at their local FedEx drop-off location. Enrolled participants were assigned to one of three types of EUA Ag-RDT (Quidel QuickVue At-Home OTC COVID-19 Test, BinaxNow COVID-19 Antigen Self-Test, or BD Veritor At-Home COVID-19 Test) and received a home delivery of 10 Ag-RDTs and 7 home collection kits for reverse transcriptase polymerase chain reaction (RT-PCR) samples. Participants were asked to perform two self-collected bilateral anterior nasal swab collections and paired testing (Ag-RDT (at home) and RT-PCR (mailed to central lab)) between study day 1 and study day 13 on 48 hour intervals, with an additional end-of-study bilateral anterior nasal swab collection for home Ag-RDT on study day Two FDA-authorized, high sensitivity RT-PCR assays were performed on each nasal swab sample received at the central lab, and an additional tiebreaker assay was performed if the two RT-PCR assays were discordant.

Measures

Ag-RDT results were based on self-reporting (Quidel QuickVue At-Home OTC COVID-19 Test, BinaxNow COVID-19 Antigen Self-Test) or an automatic reader (BD Veritor At-Home COVID-19 Test), as per EUA instructions for use. Molecular comparator RT-PCR results were based on a combination of molecular test results for detection of SARS-Cov-2 infection (Table S1), and onset of infection was defined as the day on which the molecular comparator result was positive for the first time. Cycle threshold (Ct) values for the E-gene from one RT-PCR test were used as a measure to quantify viral load. To approximate performance of Ag-RDT if a person starts testing on different days from onset of infection, we identified Days Past Index PCR Positivity (DPIPP) as different strata, for which, performance was calculated. Symptomatic or asymptomatic classification was based on presence or absence of symptoms on DPIPP for which the performance is calculated. Therefore, an individual who was asymptomatic on DPIPP 0 may become symptomatic on DPIPP 2 and vice-versa.

Statistical Analysis

Participants were eligible for inclusion in this analysis if they did not report any symptoms and had negative SARS-CoV-2 molecular and Ag-RDT tests on study day 1. Our decision to pool performance across the different tests was based on: 1) findings not shown in this report that suggested the sensitivity of different Ag-RDT was similar to each other as a function of the viral load; and 2) the study was not designed to evaluate differences in performance between the three different types of Ag-RDTs. Performance was calculated using sensitivity (rapid antigen positivity/comparator positivity) for single-day testing, two-times serial testing at 48-hour intervals, and three-times serial testing at 48-hour intervals for symptomatic and asymptomatic individuals based on day and patterns of positivity, as described in Table 1. Calculations for sensitivity were repeated with testing starting on different DPIPP. Confidence intervals for one-week sensitivity were estimated using the bootstrapping technique.7 Negative Percent Agreement was calculated as a proportion of paired tests (Ag-RDT and molecular tests performed on the same-day) based on the following formula: True Negative/[False Positive + True Negative], where True Negative refers to instances a negative molecular test was paired with a negative Ag-RDT on the same day and False-Positive refers to a negative molecular test paired with a positive Ag-RDT test. To evaluate the viral dynamics during infection, a mixed effects logistic regression model was used to predict the probability of Ag-RDT positivity based on Ct value and symptom status. Ct values, symptom status, rapid antigen testing series (one test, two-tests, or three-tests), and their three-way interactions were fixed effects, while participants’ IDs was included as a random effect. Additionally, we used the Cochran-Armitage test for trend to evaluate the associations between symptoms and Ct value categories (i.e., <20, 20-24.99, 25-29.99, 30-34.99, or 35+) among individuals who tested positive on a single rapid antigen test, two consecutive tests, and three consecutive tests, respectively. We used Bonferroni corrections (0.05/3=0.017) to adjust for multiple comparisons. All analyses were performed using R 4.1.1.8

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 1: Matrix for Calculation of Sensitivity of Rapid Antigen Tests for Detection of SARS-CoV-2 Virus in Relation to Days Past Index PCR Positivity (DPIPP)

Results

A total of 7,361 participants enrolled in the study, and 5,353 were eligible for this analysis. Of the participants eligible for analysis, 154 individuals tested RT-PCR positive for SARS-CoV-2 during the study based on a composite definition described in Table S1; 97 were without symptoms and 57 had symptoms at infection onset (Figure 1). Among the 5,199 participants who did not test positive, there were 32,998 days of paired Ag-RDT and RT-PCR testing, where comparator result was negative. Among these, 32,862 days had a concordant Ag-RDT negative result, yielding a negative percent agreement of 99.6% (95% CI: 99.5-99.7%).

Figure 1:
  • Download figure
  • Open in new tab
Figure 1: CONSORT Diagram for Test Us At Home Study to Calculate Performance of Rapid Antigen Tests for Detection of SARS-CoV-2 Virus

7,361 total participants were enrolled in the study, and 154 participants were eligible for the analysis and tested positive for SARS-CoV-2 by RT-PCR during the study period. Of these participants, 97 were asymptomatic and 57 were symptomatic on day of index comparator positive result. a= participants replaced their assigned rapid antigen tests with commercially obtained rapid antigen tests; b= dates of RT-PCR testing could not be verified based on triangulation of self-reported, shipping, and resulting data.

Performance of Ag-RDT to detect SARS-CoV-2 on day of infection onset (DPIPP: 0) was higher among symptomatic participants (59.6%, 46.7-71.4%) in comparison to asymptomatic participants (9.3%, 5.0-16.7%) (Figure 2; Table S2). Serial-testing with two Ag-RDT tests 48-hours apart (Symptomatic: 92.2%, 81.5-96.9%; Asymptomatic: 39.3%, 29.8-49.7%) and three Ag-RDT tests 48-hours apart (Symptomatic: 93.6%, 82.8-97.8%; Asymptomatic: 56.4%, 45.4-66.9%) improved performance of Ag-RDT.

Figure 2:
  • Download figure
  • Open in new tab
Figure 2: Performance of Rapid Antigen Tests for Detection of SARS-CoV-2 Virus in Relation to First Day of Molecular Positivity

Performance was calculated using sensitivity (rapid antigen positivity/comparator positivity) for single-day testing, two-times serial testing at 48-hour intervals, and three-times serial testing at 48-hour intervals for symptomatic and asymptomatic individuals based on day and patterns of positivity. Calculations for sensitivity were repeated with testing starting on different DPIPP. Error bars represent 95% Confidence Intervals. Performance of Ag-RDT to detect SARS-CoV-2 on day of infection onset was higher among symptomatic participants (59.6%) compared to asymptomatic participants (9.3%). Serial-testing with two Ag-RDT tests 48-hours apart and three Ag-RDT tests 48-hours apart improved performance of Ag-RDT within the first week of SARS-CoV-2 infection. Excluding participants with singleton RT-PCR+ results improved sensitivity of Ag-RDT among asymptomatic participants to 11.7%, 50.7%, and 74.6% based on testing one, two, or three times with Ag-RDT at 48-hour intervals, respectively, beginning on DPIPP 0.

Notably, we observed that twenty participants had a singleton RT-PCR+, defined as a positive test preceded and followed by a negative RT-PCR test within 48-54 hours. Of those with singleton RT-PCR+ tests, none tested positive on Ag-RDT, only one had symptoms on the day of the singleton positive test, and average Ct value was >35 (Figure S1). Excluding these participants did not impact sensitivity of Ag-RDT among symptomatic participants, but improved asymptomatic sensitivity to 11.7%, 50.7%, and 74.6% based on testing one, two, or three times with Ag-RDT at 48-hour intervals, respectively, beginning on DPIPP 0.

To approximate real-world scenarios, where a person may not necessarily start testing with Ag-RDT on the day of infection onset, we calculated performance separately on DPIPP 2, 4, 6, 8, and 10 (Figure 2; Table S2) to approximate scenarios where a person started serially testing with Ag-RDTs on those days. Aggregated performance of Ag-RDT for DPIPP 0-6 among all participants who were symptomatic on a given DPIPP was 82.5% (78.3-86.3%) for single-timepoint testing but with a range of sensitivity (calculated on a per DPIPP day basis) from 59.6 to 94.8%. Serial-testing using two-time testing improved sensitivity to 93.4% (90.4-95.9%) and 94.3% (91.4-97.0%) for three-time testing. Sensitivity of performing single test, two-test serial testing, and three-test serial testing for asymptomatic people was 34.4% (28.8-39.8%), 55.3% (48.2-61.6%), 68.5% (61.0-75.7%), respectively during the first week of infection (DPIPP 0-6). Excluding singleton RT-PCR positive, the first-week (DPIPP 0-6) sensitivity for asymptomatic individuals was 38.8% (32.7-45.2-%), 62.7% (57.0-70.5%), 79.0% (70.1-87.4%), respectively for testing one, two, or three times with Ag-RDT at 48-hour intervals.

Performance of Ag-RDT among symptomatic and asymptomatic participants was evaluated by Ct value to analyze the performance by viral load. The distribution of Ct values significantly differed between symptomatic and asymptomatic participants, with symptomatic participants having lower Ct values on average than asymptomatic participants at DPIPP 0 and 2 (p<0.001) (Figure 3). On the day of index PCR positivity, more than 75% of asymptomatic individuals had a Ct value 30 or higher, whereas less than 33% of symptomatic individuals had a Ct value 30 or higher. At the end of 1-week from index PCR positivity (DPIPP 6), the majority of both asymptomatic and symptomatic individuals had a Ct value less than 30. Figure 4 demonstrates the sensitivity of Ag-RDT for different Ct values based on postestimation results from a multilevel model. The sensitivity was lowest among asymptomatic participants who performed a single test for all Ct values > 20, compared to symptomatic participants who performed a single test and symptomatic and asymptomatic participants who performed two-test and three-test serial testing (Figure 4; Table S3). Two-test serial testing among symptomatic individuals and three-times serial testing among both asymptomatic individuals and symptomatic individuals demonstrated sensitivity above 80% for Ct values < 32.

Figure 3:
  • Download figure
  • Open in new tab
Figure 3: Cycle Threshold values by Day Post Index PCR-Positivity by Symptom Status

Error bars represent 95% Confidence Intervals. Symptomatic participants who tested positive by RT-PCR had significantly lower Cycle threshold (Ct) values on average than asymptomatic participants at DPIPP 0 and 2. On DPIPP 0, >75% of asymptomatic individuals had a Ct value ≥30, whereas <33% of symptomatic individuals had a Ct value ≥30. Symptomatic participants had a lower proportion of individuals with CT values ≥30 at all DPIPP. DPIPP: Day post index PCR-positive.

Figure 4:
  • Download figure
  • Open in new tab
Figure 4: Predicted Probability of Rapid Antigen Test Positivity by Symptom Status and Serial Testing Schedule

A mixed effects logistic regression model was used to predict the probability of Ag-RDT positivity based on Ct value and symptom status. Error bars represent 95% Confidence Intervals. The sensitivity was lowest among asymptomatic participants who performed a single test for all Ct values > 20. Two-test serial testing among symptomatic individuals and three-times serial testing among both asymptomatic individuals and symptomatic individuals demonstrated sensitivity above 80% for Ct values < 32.

Discussion

We report findings from the largest study to date of paired Ag-RDT and RT-PCR testing for a comparative performance evaluation of Ag-RDTs among people with and without symptoms. These results provide compelling reasons to suggest that frequency of Ag-RDT testing should be adjusted to include additional repeat testing. These data suggest an improvement in test performance when symptomatic individuals test two times 48-hours apart using Ag-RDTs. Likewise, there are notable performance improvements in asymptomatic individuals when an initial Ag-RDT test was followed by at least 2 subsequent tests at 48-hour intervals. These results should be considered in the context of our study protocol which indicated testing at 48-hour intervals, and thus these data cannot support conclusions about serial testing for time intervals shorter than 48-hours.

These findings represent a comprehensive evaluation of the time dependent performance of Ag-RDT tests among the intended use population (i.e., symptomatic and asymptomatic individuals) throughout the course of molecular test positivity. Restricting findings from our study to match observation windows from previous studies, we found similar sensitivity as found in previous studies for asymptomatic and symptomatic participants for a single-time test.9 Unlike previous reports, which used composite sampling methods and lacked sufficient longitudinal data to adequately evaluate performance of Ag-RDT from the onset of infection, we were able to approximate performance of Ag-RDT for symptomatic and asymptomatic users by comparing performance within the first week of infection, to align with the indications listed in the EUA, and to evaluate the performance of serial testing within this paradigm.10,11

We observed that more than one in ten new infections were singleton-positive, which escaped detection by rapid antigen tests. Evaluation of Ct values of these singleton-positive revealed that all of them had Ct value > 30 and average Ct count was 35. The finding of singleton RT-PCR positive testing needs to be further investigated to understand the clinical significance of this observation.

Our finding of higher Ct value associated with lower sensitivity is in line with results from a comprehensive meta-analysis encompassing data from 214 clinical studies and 112,323 samples, which demonstrated that sensitivity of rapid antigen testing deteriorated with increasing Ct values.9 We also observed that rapid antigen tests have higher sensitivity among symptomatic participants, regardless of Ct value. The finding that performance of rapid antigen tests differed with respect to Ct values between symptomatic and asymptomatic participants was unexpected, as rapid antigen test performance has often been considered to be a function of viral load.12,13 However, this may suggest that the difference in performance between asymptomatic and symptomatic participants is more than just a function of viral dynamics. It is possible that symptomatic and asymptomatic individuals differ in three other domains: 1) interpretation of results; 2) administration of tests; 3) physiology (i.e., amount of secretions available for sampling). Previous work found that symptom status was not a predictor of false negative results; however, self-interpretation of results may introduce bias.14 These three hypotheses are subject to further inquiry, as it is important to determine the role of these factors in rapid antigen test performance. Additionally, a previous report suggested that certain haplotypes of the human leukocyte antigen (HLA) loci are two to eight-fold more likely to experience an asymptomatic infection, and in a large GWAS study, this haplotype was found to be prevalent in roughly 10% of the patients.15 Similar observations have been made with Human Immunodeficiency Virus, where certain genotypes are associated with lower propensity of infection. The impact of HLA haplotype on COVID-19 infections should be investigated further. This study does have limitations. This study was conducted during the circulation of the Delta and Omicron SARS-CoV-2 variants, and future variants may warrant further investigation, especially as milder, less symptomatic variants emerge.16,17 Additionally, specimens were self-collected for RT-PCR and Ag-RDTs were self-performed. However, data has consistently shown substantial agreement between self-collected and clinician-collected anterior nasal swabs for SARS-CoV-2 testing.18,19 Further, this primary analysis does not account for differences in severity or type of symptoms. Public health implications of our findings suggest that people testing for SARS-CoV-2 should exercise caution despite an initial negative rapid antigen-test and favor mask-wearing and avoiding crowded places if they suspect they may be infected or have been exposed. Additionally, the rates of false positive results in the study were low; therefore, any Ag-RDT positive result should be considered positive without the need to retest. Further, in the context of reports of viral culture positivity more than five days after initial positive test, our findings support isolation for a longer period of time to prevent the potential of spread of SARS-CoV-2 to others.20 Further research is needed to quantitatively estimate the benefits of Ag-RDT for early detection of infection and initiation of treatment, especially in settings where access to molecular testing is limited or molecular test results are delayed. Dissemination of clear guidance for appropriate testing using Ag-RDT based on data from this study may help preserve confidence in the performance of serial Ag-RDT to detect SARS-CoV-2 virus, especially as reports of individual false negative Ag-RDT from inadequate serial testing, contrary to the tests’ intended usage and guidance from the FDA, proliferate in lay media.

Data Availability

All data produced are available online at https://github.com/soni-lab.

https://github.com/soni-lab

Competing Interest Statement

DDM reports consulting and research grants from Bristol-Myers Squibb and Pfizer, consulting and research support from Fitbit, consulting and research support from Flexcon, research grant from Boehringer Ingelheim, consulting from Avania, non-financial research support from Apple Computer, consulting/other support from Heart Rhythm Society. YCM has received research grant support to Johns Hopkins University from Hologic, Cepheid, Roche, ChemBio, Becton Dickinson, miDiagnostics, and has provided consultative support to Abbott..

Funding Statement

This study was funded by the NIH RADx Tech program under 3U54HL143541-02S2 and NIH CTSA grant UL1TR001453. The views expressed in this manuscript are those of the authors and do not necessarily represent the views of the National Institute of Biomedical Imaging and Bioengineering; the National Heart, Lung, and Blood Institute; the National Institutes of Health, or the U.S. Department of Health and Human Services. Salary support from the National Institutes of Health U54HL143541, R01HL141434, R01HL137794, R61HL158541, R01HL137734, U01HL146382 (AS, DDM), U54EB007958-13 (YCM, MLR), AI272201400007C, UM1AI068613 (YCM), U54EB027049 and U54EB027049-02S1 (CJA, RLM).

Acknowledgment

We are grateful to our study participants and to our collaborators from the National Institute of Health (NIBIB and NHLBI) who provided scientific input into the design of this study and interpretation of our results but could not formally join as co-authors due to institutional policies and to the Food and Drug Administration (Office of In Vitro Diagnostics and Radiological Health) for their involvement in the primary TUAH study. We received meaningful contributions from Drs. Bruce Tromberg, Jill Heemskerk, Dennis Buxton, Felicia Qashu, Erin Iturriaga, Jue Chen, Andrew Weitz, and Krishna Juluru. We are greatly appreciative of the contribution to this study by the numerous staff at UMass Chan, including critical support from Karen Gilliam, Mary Janet McCarthy, Amber Showers, Cynthia Kinahan, Kimberly Cantin, and Danielle Howard. We would also like to acknowledge the support provided by clinical coordinators from Threewire, Inc. We are thankful to county health departments across the country who helped with recruitment for this study by spreading the word in their networks.

Footnotes

  • Authorship updated to include FDA authors

References

  1. 1.↵
    Ricks S, Kendall EA, Dowdy DW, Sacks JA, Schumacher SG, Arinaminpathy N. Quantifying the potential value of antigen-detection rapid diagnostic tests for COVID-19: a modelling analysis. BMC Med. 2021;19(1):75. doi:10.1186/s12916-021-01948-z
    OpenUrlCrossRef
  2. 2.
    Møller IJB, Utke AR, Rysgaard UK, Østergaard LJ, Jespersen S. Diagnostic performance, user acceptability, and safety of unsupervised SARS-CoV-2 rapid antigen-detecting tests performed at home. International Journal of Infectious Diseases. 2022;116:358–364. doi:10.1016/J.IJID.2022.01.019
    OpenUrlCrossRef
  3. 3.↵
    Peeling RW, Olliaro PL, Boeras DI, Fongwen N. Scaling up COVID-19 rapid antigen tests: promises and challenges. Lancet Infect Dis. 2021;21(9):e290–e295. doi:10.1016/S1473-3099(21)00048-7
    OpenUrlCrossRefPubMed
  4. 4.↵
    Siddiqui ZK, Mihir C, Robinson ML, et al. Implementation and Accuracy of BinaxNOW Rapid Antigen COVID-19 Test in Asymptomatic and Symptomatic Populations in a High-Volume Self-Referred Testing Site. Microbiol Spectr. 2021;9(3):e01008–21. doi:10.1128/Spectrum.01008-21
    OpenUrlCrossRef
  5. 5.↵
    Prince-Guerra JL, Almendares O, Nolen LD, et al. Evaluation of Abbott BinaxNOW Rapid Antigen Test for SARS-CoV-2 Infection at Two Community-Based Testing Sites — Pima County, Arizona, November 3–17, 2020. Morbidity and Mortality Weekly Report. 2021;70(3):100. doi:10.15585/MMWR.MM7003E3
    OpenUrlCrossRef
  6. 6.↵
    Soni A, Herbert C, Pretz C, Stamegna P, Filippaios A. Finding a Needle in the Haystack: Design and Implementation of a Digital Site-less Clinical Study of Serial Rapid Antigen Testing to Identify Asymptomatic SARS-CoV-2 Infection. medRxiv. Published online August 8, 2022.
  7. 7.↵
    Davison AC, Hinkley D v. Bootstrap Methods and Their Application. Cambridge University Press; 1997. doi:DOI: 10.1017/CBO9780511802843
    OpenUrlCrossRef
  8. 8.↵
    R Development Core Team. R: A language and environment for statistical computing. Published online 2022.
  9. 9.↵
    Brümmer LE, Katzenschlager S, Gaeddert M, et al. Accuracy of novel antigen rapid diagnostics for SARS-CoV-2: A living systematic review and meta-analysis. PLoS Med. 2021;18(8):e1003735.. https://doi.org/10.1371/journal.pmed.1003735
    OpenUrlCrossRefPubMed
  10. 10.↵
    Winnett AV, Akana R, Shelby N, et al. Extreme differences in SARS-CoV-2 Omicron viral loads among specimen types drives poor performance of nasal rapid antigen tests for detecting presumably pre-infectious and infectious individuals, predicting improved performance of combination specimen antigen tests. medRxiv. Published online January 1, 2022:2022.07.13.22277513. doi:10.1101/2022.07.13.22277513
    OpenUrlAbstract/FREE Full Text
  11. 11.↵
    Chu VT, Schwartz NG, Donnelly MAP, et al. Comparison of Home Antigen Testing With RT-PCR and Viral Culture During the Course of SARS-CoV-2 Infection. JAMA Intern Med. 2022;182(7):701–709. doi:10.1001/jamainternmed.2022.1827
    OpenUrlCrossRef
  12. 12.↵
    Routsias JG, Mavrouli M, Tsoplou P, Dioikitopoulou K, Tsakris A. Diagnostic performance of rapid antigen tests (RATs) for SARS-CoV-2 and their efficacy in monitoring the infectiousness of COVID-19 patients. Scientific Reports |. 123AD;11:22863. doi:10.1038/s41598-021-02197-z
    OpenUrlCrossRef
  13. 13.↵
    Mak GC, Cheng PK, Lau SS, et al. Evaluation of rapid antigen test for detection of SARS-CoV-2 virus. Journal of Clinical Virology. 2020;129. doi:10.1016/J.JCV.2020.104500
    OpenUrlCrossRef
  14. 14.↵
    Stohr JJJM, Zwart VF, Goderski G, et al. Self-testing for the detection of SARS-CoV-2 infection with rapid antigen tests for people with suspected COVID-19 in the community. Clinical Microbiology and Infection. 2022;28(5):695–700. doi:10.1016/J.CMI.2021.07.039
    OpenUrlCrossRef
  15. 15.↵
    Augusto DG, Yusufali T, Peyser ND, et al. HLA-B*15:01 is associated with asymptomatic SARS-CoV-2 infection. medRxiv. Published online September 10, 2021:2021.05.13.21257065. doi:10.1101/2021.05.13.21257065
    OpenUrlAbstract/FREE Full Text
  16. 16.↵
    Soni A, Herbert C, Filippaios A, et al. Comparison of Rapid Antigen Tests’ Performance Between Delta and Omicron Variants of SARS-CoV-2. Ann Intern Med. Published online October 11, 2022. doi:10.7326/M22-0760
    OpenUrlCrossRef
  17. 17.↵
    Fan Y, Li X, Zhang L, Wan S, Zhang L, Zhou F. SARS-CoV-2 Omicron variant: recent progress and future perspectives. Signal Transduct Target Ther. 2022;7(1):141. doi:10.1038/s41392-022-00997-x
    OpenUrlCrossRef
  18. 18.↵
    McCulloch DJ, Kim AE, Wilcox NC, et al. Comparison of Unsupervised Home Self-collected Midnasal Swabs With Clinician-Collected Nasopharyngeal Swabs for Detection of SARS-CoV-2 Infection. JAMA Netw Open. 2020;3(7):e2016382–e2016382. doi:10.1001/jamanetworkopen.2020.16382
    OpenUrlCrossRef
  19. 19.↵
    Lindner AK, Nikolai O, Kausch F, et al. Head-to-head comparison of SARS-CoV-2 antigen-detecting rapid test with self-collected nasal swab versus professional-collected nasopharyngeal swab. European Respiratory Journal. 2021;57(4):2003961. doi:10.1183/13993003.03961-2020
    OpenUrlAbstract/FREE Full Text
  20. 20.↵
    Boucau J, Marino C, Regan J, et al. Duration of Shedding of Culturable Virus in SARS-CoV-2 Omicron (BA.1) Infection. New England Journal of Medicine. 2022;387(3):275–277. doi:10.1056/NEJMc2202092
    OpenUrlCrossRef
Back to top
PreviousNext
Posted January 23, 2023.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Performance of Rapid Antigen Tests to Detect Symptomatic and Asymptomatic SARS-CoV-2 Infection
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Performance of Rapid Antigen Tests to Detect Symptomatic and Asymptomatic SARS-CoV-2 Infection
Apurv Soni, Carly Herbert, Honghuang Lin, Yi Yan, Caitlin Pretz, Pamela Stamegna, Biqi Wang, Taylor Orwig, Colton Wright, Seanan Tarrant, Stephanie Behar, Thejas Suvarna, Summer Schrader, Emma Harman, Chris Nowak, Vik Kheterpal, Lokinendi V Rao, Lisa Cashman, Elizabeth Orvek, Didem Ayturk, Laura Gibson, Adrian Zai, Steven Wong, Peter Lazar, Ziyue Wang, Andreas Filippaios, Bruce Barton, Chad J. Achenbach, Robert L. Murphy, Matthew Robinson, Yukari C. Manabe, Shishir Pandey, Andres Colubri, Laurel O’Connor, Stephenie C. Lemon, Nisha Fahey, Katherine L Luzuriaga, Nathaniel Hafer, Kristian Roth, Toby Lowe, Timothy Stenzel, William Heetderks, John Broach, David D McManus
medRxiv 2022.08.05.22278466; doi: https://doi.org/10.1101/2022.08.05.22278466
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Performance of Rapid Antigen Tests to Detect Symptomatic and Asymptomatic SARS-CoV-2 Infection
Apurv Soni, Carly Herbert, Honghuang Lin, Yi Yan, Caitlin Pretz, Pamela Stamegna, Biqi Wang, Taylor Orwig, Colton Wright, Seanan Tarrant, Stephanie Behar, Thejas Suvarna, Summer Schrader, Emma Harman, Chris Nowak, Vik Kheterpal, Lokinendi V Rao, Lisa Cashman, Elizabeth Orvek, Didem Ayturk, Laura Gibson, Adrian Zai, Steven Wong, Peter Lazar, Ziyue Wang, Andreas Filippaios, Bruce Barton, Chad J. Achenbach, Robert L. Murphy, Matthew Robinson, Yukari C. Manabe, Shishir Pandey, Andres Colubri, Laurel O’Connor, Stephenie C. Lemon, Nisha Fahey, Katherine L Luzuriaga, Nathaniel Hafer, Kristian Roth, Toby Lowe, Timothy Stenzel, William Heetderks, John Broach, David D McManus
medRxiv 2022.08.05.22278466; doi: https://doi.org/10.1101/2022.08.05.22278466

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Infectious Diseases (except HIV/AIDS)
Subject Areas
All Articles
  • Addiction Medicine (280)
  • Allergy and Immunology (579)
  • Anesthesia (139)
  • Cardiovascular Medicine (1944)
  • Dentistry and Oral Medicine (252)
  • Dermatology (184)
  • Emergency Medicine (333)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (698)
  • Epidemiology (11100)
  • Forensic Medicine (8)
  • Gastroenterology (623)
  • Genetic and Genomic Medicine (3164)
  • Geriatric Medicine (308)
  • Health Economics (561)
  • Health Informatics (2042)
  • Health Policy (863)
  • Health Systems and Quality Improvement (782)
  • Hematology (310)
  • HIV/AIDS (682)
  • Infectious Diseases (except HIV/AIDS) (12717)
  • Intensive Care and Critical Care Medicine (707)
  • Medical Education (317)
  • Medical Ethics (92)
  • Nephrology (334)
  • Neurology (2984)
  • Nursing (164)
  • Nutrition (463)
  • Obstetrics and Gynecology (589)
  • Occupational and Environmental Health (614)
  • Oncology (1550)
  • Ophthalmology (477)
  • Orthopedics (185)
  • Otolaryngology (266)
  • Pain Medicine (202)
  • Palliative Medicine (57)
  • Pathology (402)
  • Pediatrics (912)
  • Pharmacology and Therapeutics (381)
  • Primary Care Research (355)
  • Psychiatry and Clinical Psychology (2782)
  • Public and Global Health (5590)
  • Radiology and Imaging (1093)
  • Rehabilitation Medicine and Physical Therapy (632)
  • Respiratory Medicine (759)
  • Rheumatology (338)
  • Sexual and Reproductive Health (311)
  • Sports Medicine (289)
  • Surgery (343)
  • Toxicology (48)
  • Transplantation (159)
  • Urology (132)