Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Clinical Performance of SARS-CoV-2 Molecular Testing

View ORCID ProfileDaniel A. Green, Jason Zucker, Lars F. Westblade, Susan Whittier, Hanna Rennert, Priya Velu, Arryn Craney, Melissa Cushing, Dakai Liu, Magdalena Sobieszczyk, Amelia K. Boehme, View ORCID ProfileJorge L. Sepulveda
doi: https://doi.org/10.1101/2020.05.06.20093575
Daniel A. Green
1Department of Pathology & Cell Biology, Columbia University Irving Medical Center, New York, NY
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Daniel A. Green
Jason Zucker
4Division of Infectious Diseases, Department of Internal Medicine, Columbia University Irving Medical Center, New York, NY
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lars F. Westblade
2Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, NY
3Division of Infectious Diseases, Department of Internal Medicine, Weill Cornell Medicine, New York, NY
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Susan Whittier
1Department of Pathology & Cell Biology, Columbia University Irving Medical Center, New York, NY
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hanna Rennert
2Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, NY
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Priya Velu
2Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, NY
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Arryn Craney
2Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, NY
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Melissa Cushing
2Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, NY
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Dakai Liu
5Department of Pathology and Clinical Laboratories, NewYork-Presbyterian Queens Hospital
PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Magdalena Sobieszczyk
4Division of Infectious Diseases, Department of Internal Medicine, Columbia University Irving Medical Center, New York, NY
MD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Amelia K. Boehme
6Department of Epidemiology and Neurology, Columbia University Irving Medical Center, New York, NY
PhD, MSPH
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jorge L. Sepulveda
1Department of Pathology & Cell Biology, Columbia University Irving Medical Center, New York, NY
MD, PhD
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jorge L. Sepulveda
  • For correspondence: jls2282{at}cumc.columbia.edu
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

Abstract

Molecular testing for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is the gold standard for diagnosis of coronavirus disease 2019 (COVID-19), but the test clinical performance is poorly understood. From 3/10/2020-5/1/2020 NewYork-Presbyterian laboratories performed 27,377 SARS-CoV-2 molecular assays from 22,338 patients. Repeat testing was performed in 3,432 patients, of which 2,413 had negative and 1,019 had positive first day results. Repeat-tested patients were more likely to be older, male, African-American or Hispanic, and to have severe disease. Among the patients with initially negative results, 18.6% became positive upon repeat-testing. Only 58.1% of any-time positive patients had a result of “detected” on the first test. The clinical sensitivity of COVID-19 molecular assays is estimated between 66.2 % and 95.6%, depending on the unknown number of false negative results in single-tested patients. Conversion to a negative result is unlikely to occur before 15 to 20 days after initial testing or 20-30 days after the onset of symptoms, with 50% conversion occurring at 28 days after initial testing. Forty-nine initially-positive patients converted to negative and then back to positive in subsequent days. Conversion from first day negative to positive results increased linearly with each day of testing, reaching 25% probability in 20 days. In summary, our study provides estimates of the clinical performance of SARS-CoV-2 molecular assays and suggests time frames for appropriate repeat testing, namely 15 to 20 days after a positive test and the same or next 2 days after a negative test in a patient with high suspicion for COVID-19.

Introduction

Coronavirus disease 2019 (COVID-19) which is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) (1, 2) is a global pandemic with mortality significantly higher than seasonal influenza (3). The rapid sequencing of SARS-CoV-2 genomes (1, 2) has allowed the development of multiple real-time reverse transcription-polymerase chain reaction (RT-PCR) assays that have become the gold standard to detect viral RNA and identify patients with COVID-19 as well as asymptomatic carriers. However, some patients with positive chest radiologic findings and symptoms suspicious for COVID-19 have been reported to test negatively by SARS-CoV-2 RT-PCR and require multiple consecutive tests to convert to a positive result (4, 5). There is limited information on the clinical performance characteristics of the SARS-CoV-2 molecular tests in real world cases, namely what is the predictive value of a negative result in patients suspected of COVID-19 and what is the relationship between the course of the disease, viral shedding, and positivity of the various molecular assays. During the 2020 pandemic of COVID-19, we tested 27,377 samples from 22,338 patients with SARS-CoV-2 molecular assays performed at NewYork-Presbyterian (NYP) clinical laboratories. The assays used in this patient population included six platforms based on reverse-transcription and nucleic acid amplification to detect SARS-CoV-2 specific RNA sequences. Our goal is to identify the clinical performance characteristics of SARS-CoV-2 molecular assays for diagnosis and stratification of COVID-19 patients and determine the dynamics of SARS-CoV-2 molecular assay results over time in a large dataset from multiple hospital locations in the New York City area.

Methods

Laboratory Dataset

We extracted records of all the SARS-CoV-2 tests performed at NYP laboratories from our laboratory information system (Cerner Millennium, Cerner Corporation, North Kansas City, MO) using a custom Cerner Command Language query. We tested a total number of 22,338 patients between March 10, 2020 and May 1, 2020 for SARS-CoV-2 by molecular assays at NewYork-Presbyterian affiliated hospitals and performed a total of 27,377 assays on nasopharyngeal (initially also on oropharyngeal) swab samples. The vast majority of the SARS-CoV-2 tests were reverse-transcription real time PCR assays performed with the high-throughput automated cobas 6800 (Roche Molecular Systems, Branchburg, NJ) platform (N=19,195), which went live at Columbia-NYP laboratory on 3/15/2020 and at Cornell-NYP laboratory on 3/30/2020. A smaller number of assays were performed since 3/10/2020 with in-house developed tests under FDA Emergency Use Authorization using the Rotor-Gene Q (N=1795, Qiagen, Valencia, CA) and 7500 Fast (N=89, Thermo-Fisher Scientific, Waltham, MA) instruments. From 4/3/2020 thereafter other platforms were introduced in the NYP laboratories, including the Xpert Xpress SARS-CoV-2 test analyzed on the GeneXpert (N=265) and Infinity (N=5954) platforms (Cepheid, Sunnyvale, CA), the ID Now (N=53), (Abbott, Chicago, IL) and the Panther Fusion (N=26) (Hologic Inc., San Diego, CA) instruments. Commercial molecular viral assays were validated and performed at NYP Hospital Laboratories following manufacturers’ recommendations.

Patients

The NYP laboratory SARS-CoV-2 testing dataset contains basic demographic information on each patient, including age, gender, race, and health care encounter type and location at the time of order. To better understand SARS-CoV2-2 test utilization and investigate the association with clinical information to the SARS-CoV-2 test results on a subset of patients, we merged our laboratory dataset with the Columbia COVID-19 CARE clinical database.

The COVID-CARE clinical database is an interdisciplinary database, managed by the Columbia Division of Infectious Diseases and contains data on all patients tested for SARS-CoV-2 at the New York Presbyterian West Campus (Milstein Hospital, Allen Hospital, Morgan Stanley Children’s Hospital). This interdisciplinary database contains the work of numerous divisions within the Departments of Medicine, Pediatrics, Neurology, and Obstetrics and Gynecology. The database is comprised of a general primary instrument and multiple sub-specialty instruments resulting in over 720 curated fields per patient. Data was extracted from the medical record through manual review by a team of medical students and sub-specialty fellows and attendings. Manually curated data is combined with structured data from the electronic medical records, pharmacy, and laboratory systems to complete the dataset. As of May 1st 2020, 1,624 patients had undergone manual review (Supplementary Table 1).

Clinical factors and other patient characteristic are compared between SARS-CoV-2 positive and SARS-CoV-2 negative patients in Supplementary Table 2, and between repeat-tested and single-tested patients in Supplementary Table 3. This study was approved by the Institutional Review Boards of Columbia and Cornell Universities.

Data Analysis

Categorical data was presented as frequency and percent. Continuous data was presented as mean (standard deviation) or median and range. Differences between groups were compared using Chi-square tests for categorical variables, and linear ANOVA for continuous variables. Correction for multiple testing was performed using the false discovery rate method of Benjamini-Hochberg (6). Confidence intervals for sensitivity and negative predictive values were calculated using the epiR R package. Time to event using Kaplan Meier curves were used to investigate time to conversion from initially negative to positive and from initially positive to negative. All statistical analyses were performed using the R statistical language (7).

Results

A total of 18,906 assays were performed once per patient and 8,471 assays were performed in 3,432 (15.4%) patients with more than one test completed in the span of 1 to 49 (median = 8) days between assays. There were 2,280 patients with multiple SARS-CoV-2 molecular assays performed who had a first negative, invalid, or indeterminate result and 797 patients who had an initial positive result.

Demographic and clinical characteristics

Patients with any positive SARS-CoV-2 result differed from those that were never positive for SARS-CoV-2 even with repeat testing (Supplementary Table 2). Notably, SARS-CoV-2 positive patients were more likely to be male, older than 44 years, and of self-reported African-American or Hispanic / Latino ethnicity and less likely to be Asian or Caucasian (Supplementary Table 2, all p<0.001). Not surprisingly, severe disease was more likely in SARS-CoV-2 positive patients, as indicated by a higher fatality rate (6.2% vs. 3.8%, p=0.048), presence of symptoms (91.8% vs. 72.4%, p=0.004), need for intubation (10.7% vs. 7%, p=0.026) and frequency of decompensation, as defined by an outcome of intubation, death, or discharge to hospice (13.4% vs. 8.4%, p=0.005).

To determine which factors were associated with repeat testing for SARS-CoV-2 we analyzed demographic and clinical features of repeat-tested as compared to single-tested patients (Supplementary Table 3). Our data show higher age (median = 59.9 vs. 53.4, p<0.001), higher frequency of male gender (52.2% vs. 44.3%, p<0.001), and different distribution of self-reported race and ethnicity in repeat-tested as compared to single-tested patients, with African-Americans and Hispanics / Latinos being more likely to be repeat-tested (p<0.001). At the time of the first test order, admitted patients were significantly more represented in the repeat-tested population in contrast to patients visiting the emergency department and outpatients (p<0.001). Compared with single-tested patients, repeat-tested patients were 3.2 times more likely to be admitted to the ICU (29.7% vs. 9.4%, p<0.001), 4 times more likely to be intubated (24.4% vs. 6.1%, p<0.001), 3.4 times more likely to decompensate (27.6% vs. 8.1%, p<0.001) and 1.7 times more likely to die during the observation period (8% vs. 4.7%, p=0.038).

SARS-CoV-2 Test Performance

The characteristics of the SARS-CoV-2 tests performed in repeat-tested compared to single-tested patients are described in Supplementary Table 4. The vast majority of tests were performed with the cobas 6800 and the use of the various assays was not significantly different between repeat and single-tested patients, except for a small number of patients initially tested with the Thermo-Fisher 7500 assay, which was more likely in repeat-tested patients (p<0.001, standardized Pearson residuals for repeat-tested patients = 3.9).

Among all the repeat-tested patients, 23.2% were positive on the first test (26.5% when indeterminate results were included). If a negative test was repeated on the first day, the positivity rate increased to 26.4% (29.7% with indeterminate results). Overall positivity among repeat-tested patients over the course of the study period was 39.9% in contrast to 49% for single-tested patients (p<0.001). When indeterminate results were counted as positive, 42.9% of repeat-tested patients were positive over the course of the study period in contrast to 50% of single-tested patients (p<0.001).

Indeterminate results are generally considered presumptive positive and occur when only one of two molecular targets is detected. In our repeat-tested patients with an initial result of “Indeterminate”, 54.4% ultimately had a result of “Detected”, as compared to 7% that remained indeterminate upon repeat testing and 38.6% that converted to a negative status during our study period, thus suggesting that it is acceptable to consider these patients positive. However, it is unclear whether the initially indeterminate patients that converted to negative were false positives or presented with low viral loads.

In contrast, a result of “Invalid” reflects the failure to amplify the built-in control and is likely related to poor sampling or inadequate RNA extraction usually due to high viscosity of the samples. In our study of repeat test patients, 52.7% ultimately became positive, 1.2% repeated as “Indeterminate” and 45.6% repeated as ‘Not Detected’. For the analysis of clinical sensitivity of SARS-CoV-2 molecular tests, we considered patients with invalid results as “negative”, as these results can be considered clinically false negatives in the sense that the patient may be infected and the test failed to yield a positive result. In practice, invalid results should always be repeated, preferably with a new sample, as the results are unpredictable.

Initial negative, invalid, or indeterminate SARS-CoV-2 test results were much more frequent among repeat-tested patients (Supplementary Table 4, p<0.001) and conversely, patients without a positive initial result were more likely to be repeated (21%) than initially positive patients (8%, p<0.001).

A subset of the cobas 6800 tests (N=5,343) had cycle threshold (Ct) values available for analysis. The Ct represents the PCR cycle, interpolated to two decimal digits, at which the real-time fluorescent signal crosses a pre-defined threshold for positivity. The Ct is inversely proportional to the concentration of viral RNA. The cobas SARS-CoV-2 RT-PCR assay amplifies two specific targets: Target 1 is located in the ORF1ab non-structural region that is unique to SARS-CoV-2 and Target 2 is a conserved region of the structural protein envelope E-gene common to all members of the Sarbecovirus sub-genus of coronavirus, which include SARS-CoV-2 and SARS-CoV (8, 9). The cobas assay also includes an internal control for assay performance that has no homology to the coronaviruses.

When comparing positive and indeterminate results from repeat-tested with single tested patients, there were no differences in indeterminate Ct values, which is expected as by definition the indeterminate results represent high Ct values. In contrast, the repeat-tested group (N=795) had significantly higher target 2 Ct values (median = 29.1 vs. 27.3, p<0.001 and frequency of target 2 Ct’s above 30 (45.8% vs. 36.9%, p<0.001) compared to single-tested patients (N=4,548), indicating lower viral load in the repeat-tested samples (Figure 1 and Supplementary Table 4).

Figure 1:
  • Download figure
  • Open in new tab
Figure 1:

Density distribution of cobas SARS-2-CoV2 Target 2 Ct in repeat-tested vs. single-tested patients. Top panel: Ct values from results reported as “intermediate”; bottom panel: Ct values from results reported as “detected”.

Analysis of conversion rates of repeat-tested patients

In this study, we classified repeat-tested patients in two groups according to their initial SARS-CoV-2 results: those that were initially positive, for whom repeat testing was most likely intended to ascertain recovery and non-infectiousness, and those that had an initial result of negative, indeterminate, or invalid, in whom persistent clinical suspicion for COVID-19 likely motivated repeat ordering of the test. Supplementary Table 4 shows the different clinical characteristics between repeat-tested and single-tested patients.

For the time-dependent analysis of conversion rates, we considered “initially positive” patients with any “Detected” or “Indeterminate” SARS-CoV-2 result obtained during the first calendar day of testing rather than the first positive test, to reduce bias due to nasopharyngeal sampling inadequacy (Table 1). Conversely, patients without a result of “Detected” or ‘Indeterminate” in the first day were labeled as “initially negative”. Among the 2,413 initially negative repeat-tested patients, 18.6% became positive upon repeat testing on subsequent days (Supplementary Table 5), indicating a negative predictive value of 81.3% (95% CI = 79.7 to 82.8) in this repeat-tested population with the prevalence present at the time of the study.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 1:

Number of SARS-CoV-2 molecular test results over the course of repeat testing, grouped by the highest test result on day 1.

In a separate analysis, we compared the results of the first test with the results from tests repeated the same day (Supplementary Table 5) or repeated any time after the first result (Supplementary Table 6). Among the patients with repeat testing who had initial results of “Invalid” (241), “Not Detected” (2280), or “Indeterminate” (114), 4.2% had ‘Detected’ results upon repeat testing in the first day of testing (Supplementary Table 5). This increase in positivity rate most likely results from a false-negative initial test due to pre-analytic factors such as sample inadequacy, incorrect swabbing technique, or stochastic sampling bias from low viral loads in the patient nasopharynx. After the first day of testing, repeating invalid, negative, or indeterminate results on the same day resulted in about 0.8% of additional positive results per day, for a total of 18.4% positives that were missed by the first test.

Among the 1,371 repeat-tested patients with one or more SARS-CoV-2 results of “Detected”, which can be assumed to be truly infected, only 58.1% were resulted as “Detected” on the initial test (Supplementary Table 5), and only 66.2% were reported as “Detected” on the first day (Table 1). Considering ‘Detected’ and “Indeterminate” as positive, 1,471 repeat-tested patients had one or more SARS-CoV-2 positive results over time; only 61.9% were positive on the initial test and only 69.3% had a positive result on the first day (Table 1). These data provide an estimate of the clinical sensitivity of the assay in the repeat-tested population, and establish a baseline to look at conversion rates from negative to positive.

Table 2A shows the number of patients who had an initial result of “Not Detected” on day 1 who converted to a SARS-CoV-2 positive status grouped per time after the initial test. Conversely, Table 2B shows rates of conversion to negative for patients with a status of “Detected” on day 1. Supplementary Figure 1 shows the distribution per day after onset of symptoms (a and b) or after initial testing (c to f) of conversions from positive to negative (left-side) and from negative to positive (right-side) in the two groups of repeat-tested patients. For this analysis, positive status included “Detected” and “Indeterminate” and negative status included results of “Not Detected” with “Invalid” results excluded. Among the initially positive patients, the unadjusted distributions show a peak of conversion to negative between 30 and 40 days after symptoms or around 20 days after initial testing. Less than 10% of the patients who converted to negative converted before 15 days after onset of symptoms or 10 days after initial testing. In contrast, among the patients with initially negative results who converted to positive, most conversions occurred 10 days or less after onset of symptoms, and in the first 1-3 days after initial testing.

Supplementary Figure 1:
  • Download figure
  • Open in new tab
Supplementary Figure 1:

Conversion distributions for repeat-tested patients with initial SARS-CoV-2 negative (a, c, e, and g) and positive (b, d, f, and h) results. For the analyses in a to d only the first conversion events (i.e. negative to positive - a and c - or positive to negative - b and d) were considered. N represents the total numbers of patients analyzed. The distributions in a to d are represented by density-adjusted histograms (in grey bars), kernel probability density lines (in blue), unadjusted cumulative distributions (red line) and Kaplan-Meier hazard rates (green); the timing of individual results are represented by the black marks along the x-axis. For the Kaplan Meier estimated hazard rates (e to h), in addition to the conversion events, patients were considered censored at the time of the last unchanged result were included and the number of patients at risk for conversion at each time point are shown in g and h.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 2:

Distribution of repeat tests per day after a first day result of “Not Detected” or “Indeterminate” (A) or after a first day result of “Detected” (B).

Since we cannot be certain about the conversion rates due to a significant proportion of repeat-test patients having insufficient testing performed to detect conversion (right-censoring) we used a Kaplan-Meier approach to estimate the conversion rate by day of testing (Figures 2, 3 and Supplementary Figure 1 e to h) with the following assumptions:

Figure 2:
  • Download figure
  • Open in new tab
Figure 2:

Kaplan-Meier estimate of conversion from an initially positive SARS-CoV-2 status on day 1 to a subsequent negative result.

Figure 3:
  • Download figure
  • Open in new tab
Figure 3:

Kaplan-Meier estimate of conversion rate from initially negative SARS-CoV-2 status on day 1 to a subsequent positive result.

  1. When there were multiple tests performed per patient in one day, the results were aggregated to the highest result, i.e. Detected > Indeterminate > Not Detected.

  2. For initially positive patients, an event is defined as highest result in a day of “Not Detected”.

  3. Conversely, an event is defined as a conversion from a “Not Detected”, or “Invalid” result at day 1 to “Detected” or “Indeterminate” on subsequent days.

  4. If the last test result was unchanged relative to the first day result, the patient was considered right-censored at that time.

  5. Only the first day and either the censoring day or the event day were used for each patient and indeterminate results were ignored.

The results show that the probability of converting from positive to negative in the initially positive, repeat-tested population is minimal until about 15 to 20 days after initial testing and reaches 50% at 28 days (95% CI = 27 to 29 days, Figure 2). In the initially negative repeat-tested population, the risk linearly increases every day with a 25% probability of conversion to positive of 20 days (95% CI = 16 to 23 days, Figure 3).

Since lack of conversion could be due to death of the patient, which occurred in about 8% of the repeat-tested population, we performed a competing risk analysis with death as the alternative event, using the timereg R package (10), and did not find significant differences in either the positive to negative or the negative to positive Kaplan-Meier cumulative probability of conversion (results not shown).

Interestingly, among the repeat-tested patients we have identified 11 patients that converted from an initial SARS-CoV-2 negative result to positive and back to negative over several days. We also identified 49 initially positive patients that became negative and later turned positive again. The time course of SARS-CoV-2 test results for these patients are shown on Supplementary Figure 2.

Supplementary Figure 2:
  • Download figure
  • Open in new tab
Supplementary Figure 2:
  • Download figure
  • Open in new tab
Supplementary Figure 2:

SARS-CoV-2 results from repeat-tested patients that converted from negative to positive and back to negative (A) or from positive to negative and back to positive (B). SARS-CoV-2 positive results are represented by blue dots and positive results are in red, plotted on a time scale from the date of the first test

Discussion

Our results from repeat-tested patients can be used to estimate the clinical sensitivity of the SARS-CoV-2 molecular testing in the population of patients that were selected for repeat testing, in contrast to the general population of tested patients. With all likelihood, most repeat testing on initial negative patients was performed either to follow a history of exposure, when the clinical profile did not fit the initial results, or when clinical presentation deteriorated after the initial result. Consistent with this hypothesis, repeat-tested patients were more likely to be older, male and of non-Caucasian race than single tested patients (Supplementary Table 3), consistent with the demographics of COVID-19 positive patients (Supplementary Table 2). Importantly, repeat-tested patients had worse outcomes as demonstrated by higher rates of decompensation, intubation, and mortality. Interestingly, repeat-tested patient who converted from positive to negative tended to be younger and present as outpatients, as compared to patients that remained positive, but there were no significant differences in clinical outcomes (Supplementary Table 7). In contrast, repeat-tested patients that remained negative tended to be inpatients, have longer admission duration, and were more likely to be extubated than those who converted to positive (Supplementary Table 8), suggesting that a significant number of repeat-testing in SARS-CoV-2 negative patients was performed in inpatients admitted before the pandemic or for non-COVID-19 reasons. Indeed, repeat-tested patients admitted before March 1st, 2020 were much more likely to have a persistent negative result (71.4%) than those admitted after March 1st, 2020 (39.7%, p<0.001).

In the absence of a more sensitive gold-standard, the repeat-tested patients with an eventual positive result can be considered true positives, as the analytical specificity of molecular testing is very high (3, 8, 9, 11, 12). It may be tempting to add all the 9,272 single-tested positive patients to the 1,371 repeat-tested positive patients to determine clinical sensitivity. However, we don’t know how many of the “Not detected” single-tested patients are false negatives, especially given the high frequency of asymptomatic or mildly symptomatic COVID-19 patients (13-15). Therefore, considering only the positive patients will inflate the estimated clinical sensitivity. Nevertheless, if we consider all negative results (repeated or not) to be true negative (i.e. a specificity of 100%) we can estimate the upper bound of the clinical sensitivity of a first initial result to be 94.6% (95% CI = 94.2-95%). If the test is repeated on the first day to account for nasopharyngeal sampling inadequacy, the upper boundary of the estimated sensitivity with these assumptions would be 95.6% (95% CI=95.2-96%). A lower bound of clinical sensitivity can be estimated by considering the worst case scenario that the same percentage of false negatives identified in repeat-tested patients would apply to the general population. With these assumptions, the clinical sensitivity of the SARS-CoV-2 assay can be estimated to be between 66.2% (95% CI = 63.6 to 68.7) and the above best case estimate of 95.6%.

The lower clinical sensitivity of the first day results in the ultimately positive repeat-tested patients suggests several possibilities:

  1. Viral shedding increases over time in a recently infected patient and will eventually cross the detection threshold in subsequent samples. This possibility is suggested by the lower Ct values for target 1 and particularly target 2 on the repeat-tested patients (Supplementary Table 4 and Figure 1).

  2. A significant number of samples are improperly collected, and repeat testing increases the probability of detection; this is particularly likely in the first day of testing, when suspicion may be high but the initial test results were negative or inconclusive, as shown in Supplementary Table 5.

  3. Initially tested patients were truly negative and acquired the infection nosocomially after admission.

The mean interval between an initial negative test result and the first positive in patients who converted in our study was 9.4 days (95% CI = 8.4 to 10.5 days, N = 335), which is longer than reported by Ai et al, who showed a mean interval time between initial negative to positive RT-PCR results of 5.1 ± 1.5 days with a median of 4 days (N = 15) (4). We have insufficient data to calculate incubation time as only 1.3% of the patients had symptoms after the first test, with a median time between symptoms and testing of 1.3 days. The median interval between start of symptoms and the first test in our study of 4.8 days (95% CI = 4.5 to 5.4 days) is in line with the data from Lauer et al., who calculated a median incubation time of 5.1 days (95% CI = 4.5 to 5.8 days) in patients from China (16).

The third possibility of hospital-acquired infection explaining a low rate of initial true negative result is less likely because a) patients who converted from negative to positive had shorter intervals between admission and the first test and shorter duration of admission than those who remained negative (Supplementary Table 9 and b) there were no significant differences between repeat-tested inpatients and outpatients in the average time interval between the initial negative test and the first positive test (results not shown). If there were a large number of inpatients acquiring the infection in the hospital one would expect longer intervals between admission and positivity due to the cohort of patients already admitted before the pandemic, as compared to patients recently admitted with COVID-19. Rather, these data suggest a pattern of repeated ordering in uninfected inpatients with a lower likelihood of conversion.

Our analysis of repeat-tested patients with an initial positive result, using the Kaplan-Meier estimator, indicates that conversion to a negative result is unlikely to occur until about 15 to 20 days after initial testing or 20 to 30 days after start of symptoms, when the odds ratio significantly increase (Figure 2 and Supplementary Figure 1). Conversely, when a patient is suspected of COVID-19 but the initial test is negative, repeating the test steadily increases the probably of conversion to ‘Detected’ every day (Figure 3). Repeat testing is especially likely to yield a positive result if the initial test is indeterminate or invalid. For indeterminate results, this likely reflects low levels of the virus in the sample or low viral shedding in the nasopharynx early in the course of the infection. For invalid results, this likely reflects sampling inadequacy, often due to excess mucous in the sample. Our finding of significantly higher Ct in repeat-tested patients as compared to single-tested patients supports this hypothesis.

Limitations of the study

This is an observational study without selection bias for the laboratory data, as all results were included in the analysis. However, the clinical data was restricted to a subset of patients seen at Columbia University Irving Medical Center campuses. Nevertheless, rates of positivity and demographic variables captured in the laboratory dataset were not significantly different between the other campuses, suggesting that the population in the clinical dataset is generally representative of the New York City patients tested for COVID-19. Other limitations of the study of repeat-tested patients include: 1) only 15% of the total patient’s tested had repeat testing done and 2) ordering of repeat testing was at the discretion of the health care providers and not performed according to a standard protocol, although test ordering was mostly accomplished through standardized electronic medical record order sets.

Conclusions

Our data suggest that patients with a high clinical suspicion or exposure setting suggestive of COVID-19 should be tested by a molecular SARS-CoV-2 assay and it is appropriate to repeat the test the same day or in subsequent days if the results are initially negative. Conversely, for patients with a positive SARS-CoV-2 molecular assay result, repeating the test before at least 15 days after the first test is unlikely to yield a negative result. Whether repeat positivity represents active infection or detection of nonviable viral RNA is unknown. Further studies are needed to develop predictive models of the course and outcomes of COVID-19 using well-curated demographic, clinical, and laboratory datasets.

Data Availability

The anonymized data stripped of personal health identifiers sufficient for reproducing the results in this manuscript will be shared upon request and review by NYP, Cornell and Columbia Universities.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Potential conflicts of interest

All authors have no conflicts.

Supplementary Tables

View this table:
  • View inline
  • View popup
  • Download powerpoint
Supplementary Table 1:

Numbers of unique patients in the Columbia Clinical COVID-19 dataset, NYP Laboratory SARS-CoV-2 Testing dataset, and the merged dataset.

View this table:
  • View inline
  • View popup
Supplementary Table 2:

Patient characteristics, grouped by highest SARS-2-CoV2 result in each patient. Negative indicates a result of “Not Detected” and Positive includes results reported as “Detected” or “Indeterminate”. Time-dependent clinical factors, such as age group, symptom start to test interval, and encounter type at time of test order, are counted at the time of the first test.

View this table:
  • View inline
  • View popup
Supplementary Table 3:

Patient characteristics comparing repeat-tested with single-tested patients.

View this table:
  • View inline
  • View popup
Supplementary Table 4:

SARS-CoV-2 molecular tests in repeat-tested vs. single-tested patients. N represents the number of tests in each category.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Supplementary Table 5:

Number of SARS-CoV-2 molecular test results of repeat-tested patients on day 1 of testing. Rows represent the initial test result and columns the highest test result on day 1.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Supplementary Table 6:

Number of SARS-CoV-2 molecular test results over the course of repeat testing, grouped by the initial test result on day 1.

View this table:
  • View inline
  • View popup
Supplementary Table 7:

Characteristics of initially positive repeat-tested patients comparing those that converted from positive to negative (Pos2neg) with those that remained positive (Unchanged pos).

View this table:
  • View inline
  • View popup
Supplementary Table 8:

Characteristics of repeat-tested initially negative patients comparing those that converted from negative to positive (Neg2pos), with those that remained negative (Unchanged neg) or indeterminate (Unchanged ind) patients.

Acknowledgments

We would like to acknowledge the contributions of all the dedicated medical technologists and laboratory technicians who performed testing at NYP laboratories, the efforts of the NYP Laboratory Information Team, in particular Kelvin Espinal, Sarah Russell, Dennis Camp, Yingzhe Kuang, and Bulent Oral for designing queries and providing data extracts, and the volunteers who helped gather data from electronic medical records for the COVID-19 CARE database.

References

  1. 1.↵
    Wu F, Zhao S, Yu B, Chen Y-M, Wang W, Song Z-G, Hu Y, Tao Z-W, Tian J-H, Pei Y-Y, Yuan M-L, Zhang Y-L, Dai F-H, Liu Y, Wang Q-M, Zheng J-J, Xu L, Holmes EC, Zhang Y-Z. 2020. A new coronavirus associated with human respiratory disease in China. 7798. Nature 579: 265–269.
    OpenUrlCrossRefPubMed
  2. 2.↵
    Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, Zhao X, Huang B, Shi W, Lu R, Niu P, Zhan F, Ma X, Wang D, Xu W, Wu G, Gao GF, Tan W. 2020. A Novel Coronavirus from Patients with Pneumonia in China, 2019. N Engl J Med 382: 727–733.
    OpenUrlCrossRefPubMed
  3. 3.↵
    Zhou F, Yu T, Du R, Fan G, Liu Y, Liu Z, Xiang J, Wang Y, Song B, Gu X, Guan L, Wei Y, Li H, Wu X, Xu J, Tu S, Zhang Y, Chen H, Cao B. 2020. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet 395: 1054–1062.
    OpenUrlCrossRefPubMed
  4. 4.↵
    Ai T, Yang Z, Hou H, Zhan C, Chen C, Lv W, Tao Q, Sun Z, Xia L. 2020. Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases. Radiology 200642.
  5. 5.↵
    Xie X, Zhong Z, Zhao W, Zheng C, Wang F, Liu J. 2020. Chest CT for Typical 2019-nCoV Pneumonia: Relationship to Negative RT-PCR Testing. Radiology 200343.
  6. 6.↵
    Benjamini Y, Hochberg Y. 1995. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society: Series B (Methodological) 57: 289–300.
    OpenUrlCrossRefPubMedWeb of Science
  7. 7.↵
    R Core Team. 2017. R: A Language and Environment for Statistical Computing.
  8. 8.↵
    Pfefferle S, Reucher S, Nörz D, Lütgehetmann M. 2020. Evaluation of a quantitative RT-PCR assay for the detection of the emerging coronavirus SARS-CoV-2 using a high throughput system. Euro Surveill 25.
  9. 9.↵
    Smithgall MC, Scherberkova I, Whittier S, Green DA. 2020. Comparison of Cepheid Xpert Xpress and Abbott ID Now to Roche cobas for the Rapid Detection of SARS-CoV-2. bioRxiv 2020.04.22.055327.
  10. 10.↵
    Scheike TH, Zhang M-J. 2011. Analyzing Competing Risk Data Using the R timereg Package. 1. Journal of Statistical Software 38: 1–15.
    OpenUrl
  11. 11.↵
    Cordes AK, Heim A. 2020. Rapid random access detection of the novel SARS-coronavirus-2 (SARS-CoV-2, previously 2019-nCoV) using an open access protocol for the Panther Fusion. J Clin Virol 125: 104305.
    OpenUrl
  12. 12.↵
    Zhen W, Smith E, Manji R, Schron D, Berry GJ. 2020. Clinical Evaluation of Three Sample-To-Answer Platforms for the Detection of SARS-CoV-2. J Clin Microbiol.
  13. 13.↵
    Lai C-C, Liu YH, Wang C-Y, Wang Y-H, Hsueh S-C, Yen M-Y, Ko W-C, Hsueh P-R. 2020. Asymptomatic carrier state, acute respiratory disease, and pneumonia due to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2): Facts and myths. J Microbiol Immunol Infect.
  14. 14.
    Li C, Ji F, Wang L, Wang L, Hao J, Dai M, Liu Y, Pan X, Fu J, Li L, Yang G, Yang J, Yan X, Gu B. Early Release - Asymptomatic and Human-to-Human Transmission of SARS-CoV-2 in a 2-Family Cluster, Xuzhou, China - Volume 26, Number 7—July 2020 - Emerging Infectious Diseases journal - CDC.
  15. 15.↵
    Wang Y, Liu Y, Liu L, Wang X, Luo N, Li L. Clinical Outcomes in 55 Patients With Severe Acute Respiratory Syndrome Coronavirus 2 Who Were Asymptomatic at Hospital Admission in Shenzhen, China. J Infect Dis.
  16. 16.↵
    Lauer SA, Grantz KH, Bi Q, Jones FK, Zheng Q, Meredith HR, Azman AS, Reich NG, Lessler J. 2020. The Incubation Period of Coronavirus Disease 2019 (COVID-19) From Publicly Reported Confirmed Cases: Estimation and Application. Ann Intern Med.
Back to top
PreviousNext
Posted May 08, 2020.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Clinical Performance of SARS-CoV-2 Molecular Testing
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Clinical Performance of SARS-CoV-2 Molecular Testing
Daniel A. Green, Jason Zucker, Lars F. Westblade, Susan Whittier, Hanna Rennert, Priya Velu, Arryn Craney, Melissa Cushing, Dakai Liu, Magdalena Sobieszczyk, Amelia K. Boehme, Jorge L. Sepulveda
medRxiv 2020.05.06.20093575; doi: https://doi.org/10.1101/2020.05.06.20093575
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Clinical Performance of SARS-CoV-2 Molecular Testing
Daniel A. Green, Jason Zucker, Lars F. Westblade, Susan Whittier, Hanna Rennert, Priya Velu, Arryn Craney, Melissa Cushing, Dakai Liu, Magdalena Sobieszczyk, Amelia K. Boehme, Jorge L. Sepulveda
medRxiv 2020.05.06.20093575; doi: https://doi.org/10.1101/2020.05.06.20093575

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Infectious Diseases (except HIV/AIDS)
Subject Areas
All Articles
  • Addiction Medicine (430)
  • Allergy and Immunology (756)
  • Anesthesia (221)
  • Cardiovascular Medicine (3296)
  • Dentistry and Oral Medicine (364)
  • Dermatology (280)
  • Emergency Medicine (479)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (1171)
  • Epidemiology (13381)
  • Forensic Medicine (19)
  • Gastroenterology (899)
  • Genetic and Genomic Medicine (5155)
  • Geriatric Medicine (482)
  • Health Economics (783)
  • Health Informatics (3271)
  • Health Policy (1142)
  • Health Systems and Quality Improvement (1192)
  • Hematology (432)
  • HIV/AIDS (1018)
  • Infectious Diseases (except HIV/AIDS) (14632)
  • Intensive Care and Critical Care Medicine (913)
  • Medical Education (477)
  • Medical Ethics (127)
  • Nephrology (524)
  • Neurology (4928)
  • Nursing (262)
  • Nutrition (730)
  • Obstetrics and Gynecology (883)
  • Occupational and Environmental Health (795)
  • Oncology (2524)
  • Ophthalmology (726)
  • Orthopedics (281)
  • Otolaryngology (347)
  • Pain Medicine (323)
  • Palliative Medicine (90)
  • Pathology (543)
  • Pediatrics (1302)
  • Pharmacology and Therapeutics (551)
  • Primary Care Research (557)
  • Psychiatry and Clinical Psychology (4216)
  • Public and Global Health (7507)
  • Radiology and Imaging (1706)
  • Rehabilitation Medicine and Physical Therapy (1014)
  • Respiratory Medicine (980)
  • Rheumatology (480)
  • Sexual and Reproductive Health (498)
  • Sports Medicine (424)
  • Surgery (548)
  • Toxicology (72)
  • Transplantation (236)
  • Urology (205)