Abstract
Background Virologic detection of SARS-CoV-2 through Reverse Transcriptase Polymerase Chain Reaction (RT-PCR) has limitations for surveillance. Serologic tests can be an important complementary approach. We assess the practical performance of RT-PCR based surveillance protocols, and the extent of undetected SARS-CoV-2 transmission in Shenzhen, China.
Methods We followed close-contacts of all PCR-confirmed cases detected since January 2020 and measured anti-SARS-CoV-2 antibodies in PCR-negative contacts 2-15 weeks after initial virological testing by RT-PCR using total Ab ELISA. We assessed rates of undetected infection, performance of RT-PCR over the course of infection, and characteristics of seropositive but PCR-negative individuals.
Findings The adjusted seropositivity rate for total Ab among 880 PCR-negative close-contacts was 4·1% (95%CI 2·8%-5·7%), significantly higher than among residents without known exposure to cases (0·0%, 95%CI 0·0%-1·0%). PCR-positive cases were 8·0 times (RR; 95% CI 5·3-12·7) more likely to report symptoms than the PCR-negative individuals who were seropositive, but otherwise similar. RT-PCR missed 36% (95%CI 28%-44%) of infected close-contacts, and false negative rates appear to be highly dependent on stage of infection.
Interpretation Even rigorous RT-PCR testing protocols may miss a significant proportion of infections, perhaps in part due to difficulties timing testing of asymptomatics for optimal sensitivity. Surveillance and control protocols relying on RT-PCR were, nevertheless, able to contain community spread in Shenzhen.
Funding Bill & Melinda Gates Foundation, Special Foundation of Science and Technology Innovation Strategy of Guangdong Province, and Key Project of Shenzhen Science and Technology Innovation Commission.
Introduction
Virologic detection of SARS-CoV-2 through Reverse Transcriptase Polymerase Chain Reaction (RT-PCR) is the gold standard for diagnosing infection1. Almost all diagnostic testing for COVID-19 is done using PCR-based methods, though rapid antigen tests are quickly gaining in popularity. Like all virologic tests, RT-PCR has imperfect sensitivity2-4, and patterns of viral shedding make the chance of testing positive vary over the course of infection5,6. Hence, while RT-PCR may be highly accurate at identifying those currently infectious, individuals must be tested at the right time during their infection to be detected, limiting the utility of virologic testing for measuring overall SARS-CoV-2 incidence.
Serologic tests offer an alternative approach for detecting SARS-CoV-2 infection, based on the long lasting immunologic signature it leaves, that can be detected for months, if not years, by measuring circulating antibodies against the virus. In contrast to virologic tests, serologic tests can tell if an individual has been infected long after viral clearance, though these assays also have imperfect sensitivity and specificity7.
By utilizing both tests in the same population, we can gain insights into the performance of each. In particular, we can gain understanding of the practical performance of RT-PCR based surveillance if three conditions are met (1) virologic surveillance occurred around the time of potential exposure in a population, (2) the same individuals tested by RT-PCR later received serologic tests, regardless of their virologic test result, and (3) there was a low chance of infection between the periods of virologic and serologic surveillance.
For SARS-CoV-2, Shenzhen, China is one area where all three conditions were met. The Shenzhen Center for Disease Control and Prevention (CDC) implemented extensive contact tracing among local residents during the first wave of the COVID-19 epidemic. During this wave, which started in mid-January, 2020, nearly all infections were among travellers who had been to other parts of China and their close-contacts8. After this initial epidemic wave receded, local COVID-19 cases disappeared from the region, and, as of August 1, 2020, all COVID-19 cases have been imported since February 16, 2020 9. During this period, we performed serologic testing among PCR-negative close-contacts of confirmed cases and conducted a serosurvey of local residents without known COVID-19 exposure. Here, we compare the results of these serologic analyses to those of the initial RT-PCR based surveillance to gain insight into the practical performance of RT-PCR based virologic surveillance protocols, and the extent of undetected transmission in the region over this period. Using PCR results from infected individuals, we also characterise the false-positivity rate of RT-PCR over time, both before and after symptom onset.
Methods
Study setting, design, and participants
The Shenzhen CDC implemented a surveillance program and strict quarantine policy to monitor travellers from Hubei province, where the first case of COVID-19 was detected, from early January 2020 until the lockdown in Hubei province was lifted around the end of March. Mandatory screening and quarantine of all international travellers started on March 27th. Suspected cases were also detected at local hospitals and through fever screening in neighborhoods.
Contact tracing was used to identify close-contacts of confirmed cases, defined as those who lived in the same apartment, shared a meal, travelled, or socially interacted with an index case from 2 days prior to symptom onset (see Bi et al. for a list of 21 symptoms 8). Samples were collected from suspected cases and close-contacts using nasopharyngeal swabs, and were tested for SARS-CoV-2 by RT-PCR. By protocol, RT-PCR testing was required for all close contacts at the beginning of quarantine, and release was conditional on two consecutive negative RT-PCR tests from samples collected at least one day apart. Those not contacted within 14 days of the last day of exposure to a case were tested only once. Symptomatic individuals were isolated and treated at designated hospitals regardless of RT-PCR test results for a minimum of 14 days after the last day of putative exposure. All close-contacts, asymptomatic cases testing PCR-positive for SARS-CoV-2, and travellers from Hubei (before lockdown was lifted around end of March) and abroad (after March 27th) were quarantined at centralized facilities, and monitored for 14 days after last day of putative exposure. After the lockdown in Hubei was lifted, domestic travellers were required to test negative by RT-PCR before leaving to come to Shenzhen and upon arrival in Shenzhen.
Between April 12th to May 4th, Shenzhen CDC attempted to recruit all PCR-negative close-contacts of all confirmed COVID-19 cases in Shenzhen for serological testing. Close-contacts were initially contacted by phone. RT-PCR test records before clinical diagnosis were available for all close-contacts, as well as their time and mode of putative exposure. A portion of these PCR-negative close-contacts were included in a previous study of 1,286 close-contacts that characterized epidemiology and transmission parameters of COVID-19 in Shenzhen8, hereafter referred to as the Shenzhen cohort. In addition, between April 17th and April 23rd, Shenzhen CDC conducted a serosurvey among 350 volunteers from neighborhoods where no cases were reported in two districts in Shenzhen, Luohu and Longgang, and 50 individuals from neighborhoods in Luohu where cases resided (three PCR-confirmed cases within case neighborhoods, 1·2 per ten thousand). The community sero-survey recruited the same number of volunteers in seven age groups (e.g., 0-9, …, 40-49, 50-59, and 60+ years old).
To estimate sensitivity of RT-PCR tests over the course of infection, we obtained RT-PCR results and time of sampling for a subset of infected individuals. To be included, individuals had to 1) have at least one positive RT-PCR or serologic test, and 2) have been detected through contact tracing, not through symptom-based surveillance. To be included in estimates of sensitivity of RT-PCR based on time before or after symptom onset, cases must have reported symptoms during the initial investigation. For PCR-confirmed cases, PCR-results were available up to the first positive, and potentially one additional confirmatory, RT-PCR test (from a different date).
All close-contacts and local residents gave written informed consent before participating in the serological testing. Contact tracing and RT-PCR testing is part of the continuing public health investigation of an emerging outbreak and therefore the individual informed consent was waived. The study was approved by the ethics committees of Shenzhen CDC. This work was done in support of an ongoing public health response, and hence was determined not to be human subjects research after consultation with the Johns Hopkins Bloomberg School of Public Health institutional review board.
Laboratory analyses
We assessed anti-SARS-CoV-2 antibodies in participants’ serum using commercially available total Ab, IgG, and IgM ELISA assays that detect antibodies binding SARS-CoV-2 spike protein receptor binding domain (RBD), according to the manufacturer’s instruction (Beijing Wantai Biological Pharmacy Enterprise, Beijing, China; Cat # WS1096). We excluded close-contacts sampled within two weeks of last exposure (n=295) due to the low expected sensitivity of antibody tests during this time window 10. Previously published studies reported that the manufacturer’s recommended cutoff for positivity of the total Ab ELISA had sensitivity ranging from 93% (validation cases sampled 7 to 21+ days post symptom onset) to 99% (validation cases sampled 1-43 days post symptom onset) and specificity ranging from 99% to 100% 11,12. A combined estimate of sensitivity from the two validation studies was calculated with inverse-variance weights.
We estimated the crude and adjusted sero-positive rate for total Ab, IgG, and IgM among PCR-negative close-contacts in the Shenzhen cohort, other PCR-negative close-contacts outside of the Shenzhen cohort, local residents living in neighborhoods where reported cases resided, and local residents living in neighborhoods where no cases were reported. To account for assay performance, we calculated the adjusted seropositivity rate as (proportion of positive tests + (spec − 1))/(sens + spec − 1). We further corrected the seropositivity rate for the neighborhood background infection rate, defined as the adjusted seropositivity rate of local residents living in neighborhoods where reported cases resided13-15. We based our analyses on those seropositive for total Ab unless otherwise noted.
Statistical analyses
We used Poisson regression to estimate the relative risk of developing symptoms comparing the seropositive and seronegative contacts and comparing the PCR-positive contacts and PCR-negative contacts who seroconverted. We used the Breslow Day test to assess heterogeneity in the exposure specific odds-ratio of positivity between serologic and virologic studies16. We estimated the number of close-contacts missed by RT-PCR using the adjusted seropositivity rate estimated among all tested PCR-negative close-contacts. Then, using the adjusted seropositivity rate among close-contacts in Shenzhen, we re-estimated the secondary attack rate (SAR) and effective reproductive number (R) in the Shenzhen cohort using methods previously described 8. We assumed that the PCR-negative close-contacts with serological test results were representative of all PCR-negative close-contacts in the Shenzhen cohort.
We estimated sensitivity of RT-PCR over time from symptom onset using data from both PCR positive and seropositive symptomatic contacts 8. To estimate sensitivity of RT-PCR over time from last exposure to an index case, we also included sero-converted contacts who were asymptomatic. Using an approach similar to that of Kucirka et al.5, we fitted a Bayesian logistic regression model for test sensitivity with a polynomial spline for time since symptom onset. We assessed the performance of models that incorporated polynomial splines of third to fifth degree with Widely Applicable Information Criterion (WAIC). We implemented this model in the Stan probabilistic programming language and used the rstan package to run the model and analyse outputs. We ran 6,000 iterations (four chains of 1500 iterations each with 250 warm-up iterations) and assessed convergence visually and using the R-hat statistic. All reported estimates are means of the posterior samples with the 2·5th and 97·5th percentiles of this distribution reported as the 95% confidence interval (CI). From the expected sensitivity of RT-PCR, we calculated the expected false negative rate on each day (1-sensitivity). We estimated sensitivity of RT-PCR up to a week after symptom onset because RT-PCR results after clinical diagnosis (which usually happened a few days after symptom onset and requires 1) a positive PCR result and 2) symptom onset or chest CT abnormality) were generally not available. We also fit generalized additive models (GAM) for test sensitivity as a function of the time from symptom onset (and time from last exposure to an index case) using a thin plate regression spline17 as implemented in the mgcv package in R to data form Shenzhen alone and a pooled dataset including data from Kucirka et al.5.
Role of the funding source
Shenzhen CDC had full access to all the data in the study, and all corresponding authors share the final responsibility for the decision to submit for publication. Dr. Gong, who works for the funder of the study, contributed to the initial concept generation, field protocol design (including sampling method planning), laboratory test methods planning, as well as reviewing the study results.
Results
As of August 1, 2020, 348 imported cases from other parts of China, 39 cases from abroad, and 75 locally-transmitted cases have been detected in Shenzhen (cumulative incidence 0·35 per 10,000). The majority of the locally-transmitted cases (63/75) were close-contacts of a confirmed imported case. At the time of writing, the vast majority of cases in Shenzhen (417/462), including locally-transmitted and imported cases, were confirmed during the first wave of the outbreak that ended on February 21, 2020.
From April 12 to May 4, 2020, Shenzhen CDC successfully contacted and collected serological samples from 2,345 of 4,422 PCR-negative close-contacts of COVID-19 cases. Among these, 1,175 were contacts of cases diagnosed in Shenzhen that had contact tracing records. Sera from 880 of these contacts were collected more than two weeks after exposure to an index case (Figure 1). Among the 880 close-contacts, the average age was 34·1 years (IQR=24·0, 44·0), and the majority were females (52·3%, n=460; Appendix Table 1). PCR-negative close-contacts had, on average, 32 RT-PCR tests each (IQR=2·0, 3·0; range=1,10) before the end of quarantine, and 814% (n=716) of them had more than one RT-PCR test (Figure 2). Average time of serological testing from last exposure to an index case was 71·4 days (IQR=56·0, 87·0) (Figure 1). Among these close-contacts, 27·6% (n=243) reported “frequent” contact with the index case, 28·4% (n=250) reported “moderate” contact and 44·0% (n=387) reported “rare” contact.
Forty of the 880 PCR-negative close-contacts (4·5%) were positive for total Ab antibodies; of these, 34 (3·9%) tested positive for IgG and 16 (1·8%) for IgM. Adjusting for assay performance 11,12, we estimated a seropositivity rate of 4·0% (95%, CI 2·8% to 5·6%, Table 2). The seropositivity rate among PCR-negative close-contacts in Shenzhen was significantly higher than local residents without known exposure to cases; only one person (0·29%, 1/350) was seropositive in neighborhoods where no cases resided and no one (0/50) was seropositive in neighborhoods where reported cases resided.
Among PCR-negative close-contacts sampled within 60 days from the last day of exposure to a known case, the unadjusted seropositivity rate was 2·9% (7/241), it was 5·6% (29/546) for those sampled 61-90 days after, and 4·3% (4/93) for over 90 days (p-value=0·33; Table 1). The overall proportion seropositive among PCR-negative close-contacts did not differ significantly by age or sex (Table 1).
Only three of the 40 (7·5%) seropositive contacts who were PCR-negative reported symptoms between the last date of putative exposure and end of quarantine (all three had fever, two had signs of lung infection, and one had nausea and headache) (Figure 2, Table 1). Nevertheless, seropositive but PCR-negative contacts were more likely to report symptoms than those who were both PCR-negative and seronegative (RR = 7·9; 95%CI, 1·7 to 27·2; p-value = 0·0023). In the Shenzhen cohort, PCR-positive cases were 8·0 times (RR, 95% CI, 5·3 to 12·7, p-value <0·0001) more likely to report symptoms than the PCR-negative individuals who seroconverted (366/391 vs. 2/17). In this PCR-negative group, there was no significant difference in the number of PCR-tests seropositive and seronegative contacts received (seropositives had on average 0·81 more tests, 95% CI, -0·041 to 1·7).
Using the adjusted seroprevalence of 4·1% (95%CI, 2·9% to 5·7%) among PCR-negative close-contacts (Table 2), we estimated that RT-PCR missed 36% (95%CI, 28% to 44%; 48 infections) of all infected close-contacts in the Shenzhen cohort. Adjusting for these missed infections increases our estimate of the effective reproductive number by 34% to 0·56 (95%CI, 0·45 to 0·67). Overall, our estimates of the secondary attack rate and household secondary attack rates increased to 108% and 159%, compared with previous estimates of 6·6% and 11·2% respectively.
Among the 288 members of the previously published Shenzhen cohort that were included in this serosurvey, we compared risk factors for seropositivity in PCR-negatives with the originally published results (Figure 1, Appendix Table 2). As noted above, those testing PCR-positive were eight times as likely to report symptoms than their PCR-negative, seropositive counterparts. However, in the Shenzhen cohort there was no significant difference in the odds-ratio of testing positive for any exposure between the serologic and virologic studies.
We examined the variability in the false negative rate of PCR-based testing over the course of infection using RT - PCR results from 60 symptomatic cases who were either diagnosed with COVID-19 by RT-PCR (n=57) or had only PCR-negative test but later tested seropositive (n=3) (Figure 3a). Seven cases had their first RT-PCR test before symptom onset and 20 on the day of symptom onset. Based on the best fit polynomial spline model (three degrees of freedom, Appendix Table 3) we estimate the probability of having a false negative results, to be 34% (95% CI, 21% to 51%) on the day of symptom onset, decreasing to a low of 11% (95% CI, 5% to 21%) four days after symptom onset. Uncertainty in the false negative rate is high in the days prior to symptom onset, but we estimate a smooth increase in the probability of a false negative rate as we move earlier in the course of infection, reaching 100% at 5 days prior to symptom onset; but 95% confidence intervals include false negative rates lower than 75% even 8 days prior to symptom onset.
Discussion
The specific circumstances of the initial SARS-CoV-2 epidemic in Shenzhen, China allowed us to use serologic data to gain additional insight into the epidemiology of the virus and the practical performance of RT-PCR as a surveillance tool. We find that 4·5% of PCR-negative close-contacts in Shenzhen were seropositive for anti-SARS-CoV-2 antibodies. Thus, despite a rigorous testing regime, virologic testing failed to detect 30-40% of infections. Even with these limitations, the serologic evidence suggests that the overall control program, which relied on RT-PCR to clear people from isolation or quarantine, was successful in containing the virus; as the rate of seropositivity outside of close-contacts was virtually zero. Those testing PCR-negative but seropositive were significantly less likely to have symptoms than those testing PCR-positive, potentially reflecting difficulties in appropriately timing virologic testing when there is no outward indication of the timing of viral shedding. Consistent with this latter point, we found considerable variation in the chance of an infected individual testing negative over the course of their infection.
It may be that RT-PCR test results are correlated with transmissibility, but we were unable to assess that in this study. Strict quarantine practices were in place, requiring all close-contacts to be quarantined in centralized facilities for about 2 weeks after exposure. Critically, individuals remained in quarantine regardless of symptoms or test results. The vast majority (92·5%) of individuals who were seropositive, but PCR-negative, reported no symptoms. While this may stem from correlations between viral shedding and development of symptoms, it also may highlight a challenge in using virologic testing for asymptomatic surveillance. The period in which RT-PCR testing is highly sensitive is relatively short5,6, and sensitivity peaks around the time of, or shortly after, symptom onset. Hence, without symptoms as an indicator of when to test, it may be difficult to capture patients in this period. This phenomena is not unique to SARS-CoV-2, and studies of influenza and other acute respiratory viruses have shown serologic attack rates 2-3 times virological attack rates18,19. Therefore, while RT-PCR is an invaluable diagnostic tool, it has important limitations as a tool for surveillance or an outcome in risk factor studies.
The study has important limitations. We were unable to obtain serologic data on PCR-positive cases, so we could not estimate the practical sensitivity of the serologic tests. Sample size was limited, particularly for estimating risk factors in the Shenzhen cohort, and only 20% of PCR-negative contacts both could be recontacted and had detailed data from the initial contact tracing. The serum samples were collected between two weeks to four months after the last day of exposure to a known case. Although we expect most cases to seroconvert in the first month after exposure, time to sampling may not have been enough for some close-contacts to seroconvert. While the serologic tests have imperfect specificity, seroprevalence among PCR-negative close-contacts were much higher than the seroprevalence among those without known exposure to cases, lending confidence that the observed undetected infections are real. Seropositivity rate among the subset in the Shenzhen cohort is slightly higher than the rate among all close-contacts in Shenzhen (5·4% vs. 4·0%), possibly due to the more frequent self-reported exposure between index cases and contacts in the cohort. The extent of undetected transmission estimated using the lower seropositivity rate in Shenzhen may provide an underestimate of the true extent.
We cannot completely rule out the possibility that secondary exposure to cases other than their index cases led to seroconversion in close-contacts because we did not ascertain recent travel history of these individuals. However, the epicenter of COVID-19 in China was under lockdown from January 23 to March 25th and mandatory RT-PCR testing of all international travellers arriving in Shenzhen was enforced starting March 27th. As a result, the chance of these individuals travelling to high risk areas before serological screening that were performed in April was low. Given the low seropositivity rate in community samples without direct exposure to a known case, we believe seroconversion among close-contacts was most likely due to exposure to the index case. We cannot rule out infection by the index case after initial quarantine, as there have been reports of viral shedding after multiple PCR-negative tests20.
Overall, this study gives important insights into the practical limitations of RT-PCR based virologic surveillance for SARS-CoV-2. While virologic testing should remain the bedrock of disease control programs and patient diagnostics, it, like all tools, is imperfect. Hence, it is essential that we weigh all the evidence when making clinical and public health decisions, and not rely on test results alone. While serologic testing is an important supplement to RT-PCR for broad scale surveillance and scientific study, it is a fundamentally different tool and cannot replace virologic testing. Innovations to both improve accuracy of virologic testing, and allow for more frequent less invasive testing are, therefore, sorely needed. Still, the Shenzhen experience highlights that even with the imperfect tools we have available, SARS-CoV-2 control is possible through rigorous and aggressive surveillance, isolation and quarantine combined with broader public health action.
Data Availability
Summary data are available upon reasonable request
>Supplemental material
Acknowledgement
We thank all patients, close contacts, and their families involved in the study, as well as the front-line medical staff and public health workers who collected these important data. The study was funded by Bill & Melinda Gates Foundation (INV-006376), Special Foundation of Science and Technology Innovation Strategy of Guangdong Province of China (N0.2020B1111340077), and Key Project of Shenzhen Science and Technology Innovation Commission, Shenzhen, China (202002073000003).
Footnotes
corrected author list