Evaluation of the Access Bio CareStartTM rapid SARS-CoV-2 antigen test in asymptomatic individuals tested at a community mass-testing program in Western Massachusetts

Background: Point-of-care antigen-detecting rapid diagnostic tests (RDTs) for Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) represent a scalable tool for SARS-CoV-2 infections surveillance. Data on their performance in real-world community settings is paramount for their implementation. Method: We evaluated the accuracy of CareStartTM COVID-19 Antigen test (CareStart) in a testing site in Holyoke, Massachusetts. We compared CareStart to a SARS-CoV-2 reverse transcriptase quantitative polymerase chain reaction (RT-qPCR) reference, using anterior nasal swab samples. We calculated the sensitivity, specificity, and expected positive and negative predictive values at different SARS-CoV-2 prevalence estimates. Results: We performed 666 tests on 591 unique individuals. 573 (86%) were asymptomatic. There were 52 positive tests by RT-qPCR. The sensitivity of CareStart was 49.0% (95% Confidence Interval (CI): 34.8 - 63.4) and specificity was 99.5% (95% CI: 98.5 - 99.9). Among positive RT-qPCR tests, the median cycle threshold (Ct) was significantly lower in samples that tested positive on CareStart. Using a Ct [≤] 30 as a benchmark for positivity increased the sensitivity to 64.9% (95% CI: 47.5 - 79.8). Conclusions: CareStart has a high specificity and moderate sensitivity. The utility of RDTs, such as CareStart, in mass implementation should prioritize use cases in which a higher specificity is more important.


52
The Coronavirus disease 2019  pandemic caused by the Severe Acute 53 Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) is the most significant infectious disease 54 pandemic in the last century (1, 2). In addition to preventive measures such as social distancing, 55 mask wearing, and vaccination, pillars of pandemic control rely on tools to rapidly identify cases in detecting SARS-CoV-2 infection in asymptomatic carriers. In fact, several independent 77 evaluations demonstrate the decreased sensitivity of antigen-detecting RDTs in asymptomatic 78 RT-qPCR positive individuals compared to those with symptoms (14-17). In the United States, 79 studies thus far have focused on 3 RDTs: Quidel Sofia™ (8,18), BD Veritor™ (13,19) and Abbott 80 BinaxNOW™ (15,16,20). On March 31 st , 2021, the FDA also authorized these tests for home 81 use, raising concerns about misinterpretation of false negative results (21). Therefore, evidence 82 to establish their performance characteristics to guide their implementation in real-world settings 83 is even more urgent now.

84
In this study, we evaluated the Access Bio CareStart™ COVID-19 RDT (CareStart), a 85 chromatographic antigen-detecting lateral flow immunoassay that received EUA by the FDA (12, 86 17). We evaluated CareStart in asymptomatic and mildly symptomatic individuals presenting for 87 routine testing at one of the 'Stop the Spread' free community testing sites in Holyoke,

88
Massachusetts. Public health messaging for testing at these community testing sites targeted 89 asymptomatic individuals. We evaluate the sensitivity, specificity, and positive (PPV) and negative week. Individuals who presented to the site during testing hours were approached by our research 101 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted June 20, 2021. ;https://doi.org/10.1101https://doi.org/10. /2021 staff who explained the nature of the study, risks, benefits, and answered any questions before 102 inviting individuals to participate in the study. Verbal consent was obtained from participants to 103 collect a second anterior nasal swab as well as from guardians of minors below 18 years of age, 104 from whom verbal assent was also obtained. The participants were treated in accordance with

123
Both anterior nares were swabbed 2 times (5 rotations in each nostril), once for RT-qPCR testing 124 and once for the RDT sample. For practical reasons, the swabs for RT-qPCR and RDT were not 125 always collected in the same order. Both samples were placed inside closed test tubes. The RT-

126
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted June 20, 2021. ;https://doi.org/10.1101https://doi.org/10. /2021 qPCR sample was transported to the Broad Institute at the Massachusetts Institute of Technology.

127
The second anterior nasal swab sample was transported to a nearby testing station and the RDT 128 was performed within an hour of sample collection. The RT-qPCR testing results were interpreted 129 according to the publicly available rubric for the Broad Institute COVID-19 testing program:  CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10. 1101/2021 We calculated sensitivity, specificity, PPV and NPV of the RDT from 2 x 2 contingency tables 152 using RT-qPCR as the gold standard reference. Tests with undetermined CareStart or RT-qPCR 153 results were excluded from these calculations (n=35; 5.2% of all tests). Sensitivity and specificity 154 were further stratified and compared by presence of symptoms and quantitative Ct values. Median

155
Ct values were compared using the non-parametric unpaired Mann-Whitney U test. 95% Pearson-

156
Clopper confidence intervals (CI) were calculated for sensitivity and specificity estimates. Since

157
RDTs have been reported to have high accuracy among symptomatic individuals (8, 15-17), we 158 also tested whether presence of symptoms would increase the sensitivity of the CareStart RDT.

162
We performed 666 CareStart RDTs from participants who provided verbal consent at the walk-up 163 testing site (Table 1 and Figure 1)

174
The study staff evaluated the usability of the CareStart devices. All tests showed a positive 175 control band, indicating they were valid. However, plastic in some vial caps was slightly 176 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021.

180
To determine the accuracy of the CareStart RDT, we calculated the concordance between 181 the RDT and RT-qPCR (    CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101/2021.06.17.21259109 doi: medRxiv preprint (24), these data suggest that applying a more stringent Ct value threshold moderately improves 202 the sensitivity of the CareStart RDT.

203
Positive and negative predictive values of diagnostic tests depend on the prevalence of 204 infections in a population, where a higher prevalence increases the PPV at the expense of the 205 NPV (26). We calculated the PPV and NPV values as a function of prevalence rates up to 10%,

206
where the PPV steeply dropped in prevalence rates lower than 5% (Figure 4). At a sensitivity of 207 49% and specificity of 99.5% (Table 3), the PPV of CareStart was 49.7% at a SARS-CoV-2 208 infection prevalence of 1%, and 91.6% at a prevalence of 10%. In contrast, the NPV was 99.5% 209 at a prevalence of 1%, and 94.6% at a prevalence of 10%. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021.

267
In laboratory studies, lower RT-qPCR Ct values were shown to correlate with increased recovery

276
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021.

285
Therefore, this limited sample supports the potential usefulness of serial rapid antigen testing in 286 detecting recent infections to implement containment measures and merits further study (38).

287
Importantly, the public health benefits of serial testing with RDTs should be studied further.

288
There are several advantages to rapid testing. CareStart has a lower turnaround time than

300
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021. ;https://doi.org/10.1101https://doi.org/10. /2021 A few factors may have compromised the performance characteristics of CareStart in this 301 study. We had limited control over or ability to monitor the order by which the two bilateral nasal 302 swabs were collected because of embedding the study in a 'real world' testing program. It is 303 possible that performing the PCR swab first may decrease the available viral load for the antigen 304 test. However, a recent evaluation of the Abbott BinaxNOW suggested that the order of swabs    CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021.  . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021.  . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021.

335
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

338
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021. ;https://doi.org/10.1101https://doi.org/10. /2021   CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021. ;https://doi.org/10.1101https://doi.org/10. /2021  CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021. ;https://doi.org/10.1101https://doi.org/10. /2021 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101/2021.06.17.21259109 doi: medRxiv preprint . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101/2021.06.17.21259109 doi: medRxiv preprint . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101/2021.06.17.21259109 doi: medRxiv preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021.  . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021.   CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021. ; https://doi.org/10.1101/2021.06.17.21259109 doi: medRxiv preprint Repeat testers . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted June 20, 2021. ;

Cough
No 2  . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted June 20, 2021.