Performance of three SARS-CoV-2 immunoassays, three rapid lateral flow tests and a novel bead-based affinity surrogate test for the detection of SARS-CoV-2 antibodies in human serum

For the control of immunity in COVID-19 survivors and vaccinated subjects there is an urgent need for reliable and rapid serological assays. Based on samples from 63 COVID-19 survivors up to seven months after symptom onset, and on 50 serum samples taken before the beginning of the pandemic, we compared the performance of three commercial immunoassays for the detection of SARS-CoV-2 IgA and IgG antibodies (Euroimmun SARS-COV-2 IgA/IgG, Mikrogen recomWell SARS-CoV-2 IgA/IgG, and SERION ELISA agile SARS-CoV-2 IgA/IgG) and three rapid lateral flow (immunochromatographic) tests (Abbott Panbio COVID-19 IgG/IgM, NADAL COVID-19 IgG/IgM, and Cleartest Corona 2019-nCOV IgG/IgM) with a plaque-reduction neutralization test (PRNT50) representing the gold standard. In addition, we report and validate a novel, non-commercial flow cytometry bead-based surrogate test. 57 out of 63 PCR-confirmed COVID-19 patients (90 %) showed neutralizing antibodies. The sensitivity of the seven assays ranged from 7.0 % to 98.3 %, the specificity from 86.0 % to 100.00 %. Only one commercial immunoassay showed a sensitivity and specificity of greater than 98 %. These data indicate abundant interassay variability.


Abstract 23
For the control of immunity in COVID-19 survivors and vaccinated subjects there is an urgent 24 need for reliable and rapid serological assays. 25 Based on samples from 63 COVID-19 survivors up to seven months after symptom onset, and 26 on 50 serum samples taken before the beginning of the pandemic, we compared the 27 performance of three commercial immunoassays for the detection of SARS-CoV-2 IgA and 28 IgG antibodies (Euroimmun SARS-COV-2 IgA/IgG, Mikrogen recomWell SARS-CoV-2 29 IgA/IgG, and SERION ELISA agile SARS-CoV-2 IgA/IgG) and three rapid lateral flow 30 is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 8, 2021. ;https://doi.org/10.1101https://doi.org/10. /2021 Introduction 41 The novel severe acute respiratory syndrom coronavirus 2 (SARS-CoV-2) causing coronavirus 42 disease  was first reported in China at the end of 2019 and since then has caused 43 a worldwide pandemic. As of February 7, 2021, more than 105 million cases and 2.2 million 44 deaths have been reported attributed to COVID-19 worldwide (1). 45 SARS-CoV-2 is mainly transmitted from person to person through respiratory droplets and 46 aerosols, although indirect transmission via contaminated objects or surfaces may also be 47 possible (2, 3). The virus enters host cells via the angiotensin-converting enzyme 2 (ACE2) (4, 48 5), which is highly expressed on lung alveolar epithelial cells and small intestinal epithelial cells 49 (6). Typical clinical symptoms of COVID-19 are olfactory or gustatory dysfunction, fever, 50 fatigue, dry cough, shortness of breath, headache, and joint pain. Severe COVID-19 cases 51 can result in acute respiratory distress syndrome, septic shock, metabolic acidosis, coagulation 52 dysfunction, and multiple organ dysfunction syndromes (7). In order to control the spread of 53 the virus, many countries have implemented stringent non-pharmaceutical interventions as 54 well as extensive laboratory diagnostic tests of symptomatic but also asymptomatic people. 55 A rapid and early laboratory diagnosis is critical to diagnose infection and control transmission. 56 point-of-care diagnostics. Based on the currently available studies, the sensitivity of ELISA 60 assays for antibody detection increases during the second week after the onset of symptoms 61 (10) when infectiosity is most likely lost and is therefore not eligible for diagnosis of active 62 COVID-19 disease. 63 Nevertheless, antibody testing is needed to determine if persons have been exposed to SARS-64 CoV-2, in patients with potential post-COVID-19-symptoms as well as epidemiological 65 investigations. In patients with doubtful PCR results, antibody testing helps to distinguish 66 between active SARS-CoV-2 infections, persistant PCR positivity after an undiagnosed SARS-67 CoV-2 infection or false positive PCR results (11). It also allows the identification of suitable 68 convalescent plasma donors and may become important for the determination of post-69 vaccination immunity. For these purposes, especially a high specificity is important to avoid 70 "false immune" cases. 71 Virus neutralization assays remain the gold standard for determining antibody efficacy but are 72 complex, difficult to standardize, and time-consuming. In contrast, enzyme-linked 73 immunosorbent assays (ELISA) are feasible in a routine laboratory setting and cheap. To 74 support decision making on the employment of antibody testing for either diagnostic or 75 population screening, we present a detailed comparison of serological COVID-19 assays to 76 virus neutralization assays. 77 Several publications have assessed sensitivities of various ELISAs for laboratory use and 78 lateral flow assays for point-of-care use as proportion of previously PCR positive patients at 79 least 14 days after the PCR and found sensitivities between 15 % to 100 % (12-15), but only 80 few have evaluated these assays against tests that directly measure virus neutralization (16-81 19 is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 8, 2021. ; https://doi.org/10.1101/2021.02.07.21251062 doi: medRxiv preprint ~200/well. Serum dilutions were mixed 1:1 with pseudoparticles and incubated for 45 minutes 140 at 37°C prior to addition to the preplated Vero cells and incubation at 37 °C for 16 hours. The 141 number of GFP+ viral foci was counted using a fluorescent microscope. The 50% pseudovirus 142 neutralization titre (pVNT50) was reported as the interpolated reciprocal of the dilution yielding 143 a 50% reduction in fluorescent viral foci. Neutralization against GFP-expressing recombinant 144 VSV was assayed in parallel to exclude unspecific inhibition of infection. In case of an inhibition 145 greater than 30 % in the inhibition control, PRNT50 titers were read in relation to the PFU in 146 the inhibition control. Neutralization testing was repeated two times in sera with a SARS-CoV-147 2 independent neutralization greater than 50 % as well as in sera negative in the neutralization 148 testing but with at least one borderline or positive ELISA test. The median titer was then 149 selected for further analysis. PRNT50 ≥ 20 were considered positive. Würzburg, Germany). The Euroimmun assay is based on S1 antigen, Mikrogen recomWell is 157 based on the nucleocapsid (N) protein as antigen and SERION ELISA agile is coated with both 158 S1 and S2 and the nucleocapsid. The assays were performed following the manufacturers' 159 instruction. The following thresholds were used: In the Mikrogen assays, samples with a 160 concentration of < 20 U/ml were assessed as negative, samples with a concentration of ≥ 20 161 U/ml but < 24 U/ml were counted as borderline, samples with a concentration of ≥ 24 U/ml 162 were counted positive. For Euroimmun's IgA and IgG assays, extinction ratios were calculated 163 out of specimen extinction and calibrator extinction. Ratio values < 0.8 were considered as 164 negative; those with a ratio ≥ 1.1 were considered as positive and those with a ratio ≥ 0.8 to < 165 . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 8, 2021. ; https://doi.org/10.1101/2021.02.07.21251062 doi: medRxiv preprint 1.1 were recorded as borderline according to the manufacturer's recommendations. The 166 thresholds in the SERION assay were < 10 U/ml for negative results, ≥ 10 U/ml but < 14 U/ml 167 for IgA and ≥ 10 U/ml but < 15 U/ml for IgG antibodies as borderline and > 14 U/ml for IgA and 168 > 15 U/ml for IgG as positive. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 8, 2021. ; https://doi.org/10.1101/2021.02.07.21251062 doi: medRxiv preprint followed by incubation with 1:100 dilution of serum in PBS / 0.1 % BSA / 0.02% NaN3. After 193 extensive washing, bead-bound antibodies were detected with Goat-anti-human IgG Alexa 647 194

Statistical analysis 210
Data analysis was performed using Microsoft Excel 2016, IBM SPSS Statistics 26, and 211 GraphPad Prism 8. Differences in symptoms between patients with and without neutralizing 212 antibodies as well as differences in test sensitivity were analyzed using two-tailed Fisher's 213 exact test. Differences in the time interval between PCR result and serum collection were 214 compared using Student's t-test. Spearman's rank correlation coefficient rho was calculated to 215 determine the correlation of different tests. Two-tailed significance was calculated using exact 216 permutation based probabilities. For the calculation of specificity and sensitivity, borderline 217 results of the serological tests were counted as positive. Specificity was calculated as the 218 proportion of negatively detected samples in the negative sera from 2018, sensitivity was 219 . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 8, 2021. ; https://doi.org/10.1101/2021.02.07.21251062 doi: medRxiv preprint

Patient selection 229
Out of 68 participating patients, 63 were included in the study. Five patients did not meet the 230 inclusion criteria: Three patients reported that their diagnosis was not based on a PCR result 231 but on SARS-CoV-2 antibody rapid testing. Neutralizing antibodies were not found in these 232 three patients, but various commercial assays were positive for SARS-CoV-2 antibodies. The 233 sera of two patients were taken 3 and 13 days, respectively, and thus too early after their first 234 positive PCR result. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

Comparison of commercial immunoassays with PRNT50 284
Results of Euroimmun SARS-COV-2 IgA/IgG, Mikrogen recomWell SARS-CoV-2 IgA/IgG and 285 SERION ELISA agile SARS-CoV-2 IgA/IgG were plotted against the titers observed in the 286 neutralization assays (Fig. 3). is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

299
The sensitivities of the IgA assays to detect neutralizing antibodies ranged from 31.6 % 300  Table 1.  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 8, 2021. ;https://doi.org/10.1101https://doi.org/10. /2021 While the sensitivities of Mikrogen IgA (p < 0.001), Mikrogen IgG (p = 0.002) and SERION IgA 316 (p = 0.008) were significantly lower compared to the best performing assay SERION IgG, the 317 differences between Euroimmun IgA / IgG and SERION IgG were not significant (p = 0.21). All

Rapid antibody testing (Lateral flow assays) 327
The sensitivities of the IgM assays to detect neutralizing antibodies ranged from 7.0 % (Abbott) 328 to 66.7 % (NADAL), the specificities from 91.1 % (Cleartest) to 100.0 % (Abbott and NADAL). 329 The sensitivities of the IgG assays ranged from 84.2 % (Abbott) to 89.5 % (Cleartest), the 330 specificities from 95.6 % (Cleartest) to 100.0 % (Abbott and NADAL). Detailed data and 331 confidence intervals are shown in Table 2  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

Bead-based surrogate test 345
The sensitivity of the bead-based surrogate test for neutralizing antibodies was 71.9 % (95% 346 CI: 59.2 -81.9 %), the specificity 100 % (95 % CI: 92.9 -100.0 %). The total bead score is 347 correlated with the inhibition corrected PRNT50 (rho = 0.414; p = 0.001, Fig. 4).  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 8, 2021. ; https://doi.org/10. 1101 showed borderline results in the IgG assay and the Abbott IgG showed a positive result, in one 359 patient only Nadal IgM/IgG was positive. Interestingly, one patient had positive IgG antibodies 360 in the Mikrogen, SERION, and Abbott assay as well as positive IgM and borderline IgG in the 361 Euroimmun assay. The bead-based surrogate test was negative in all of these six patients. 362 363

Antibody dynamic 364
While the IgA antibodies determined by the Mikrogen ELISA assay were significantly 365 negatively correlated with the time between the first positive PCR and the serum sampling 366 (rho = -0.313; p=0.01), this was not the case for antibodies measured by other ELISAs, 367 neutralization testing and the flow cytometry bead-based test. 368 We followed the seroconversion against SARS-CoV-2 in one representative patient with 369 moderate symptoms using the neutralization assay, all six ELISA systems, and the bead- IgG occurred later than that for IgA and positive IgG levels were detected from day 11 on with 374 the SERION IgG ELISA and the Mikrogen IgG assay, whereas the Euroimmun IgG assay failed 375 to detect IgG antibody levels until day 24. Interestingly both IgA and IgG levels decreased after 376 day 17 in all but the Euroimmun IgG assay, where the levels of detected antibodies even 377 continued to increase. Neutralizing antibodies could be detected from day 11 on with a peak 378 on day 14. The flow cytometry bead-based assay gave ambiguous results.  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 8, 2021. ;https://doi.org/10.1101https://doi.org/10. /2021 PRNT50: 50 % plaque reduction neutralization test. Method described in the methods section.

383
Euroimmun: Euroimmun SARS-COV-2 IgA / IgG (Euroimmun, Lübeck, Germany) 384 Neuried,Germany) 385 SERION: Würzburg,Germany)  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 8, 2021. ;https://doi.org/10.1101https://doi.org/10. /2021 Discussion 389 As a large variety of serological assays for the detection of anti-SARS-CoV-2 antibodies are 390 now becoming available, proper assay validation is required to inform decisions on the use of 391 these assays and the interpretation of test results for specific clinical and public health care 392 demands. In this study we determined the specificity and sensitivity of three commercially 393 available immunoassays and three rapid lateral flow tests for the detection of SARS-Cov-2 394 antibodies in comparison with a neutralization test. Neutralization assays remain the gold 395 standard for determining antibody efficacy. 90 % of the PCR-confirmed COVID-19 patients 396 investigated in this study showed neutralizing antibodies against the SARS-CoV-2 spike 397 protein up to seven months after symptom onset. This is in line with literature data ranging 398 from 77 % (19) to 96 % (30). Neutralizing antibody production seems to be correlated with 399 severity of disease (30) and one study found neutralizing antibodies only in 12.5 % of 400 asymptomatic SARS-CoV-2 positive patients (31). Both asymptomatic patients included in this 401 study did not show neutralizing antibodies either, and it is open to be discussed. whether the 402 original PCR results were false positive. The persistance of IgG antibodies has yet to be 403 defined. However IgG levels seem to persist for at least seven months, as detected with 404 different commercial assays as well as the bead-based surrogate test and the pseudovirus 405 neutralisation assay. In comparison, SARS-CoV-1 patients were shown to maintain IgG 406 antibodies for an average of two years, and significant reduction of IgG levels and titres 407 occurred in the third year (32). 408 Sensitivity of the seven assays to detect sera with neutralizing activtity ranged from 7.0 % to 409 98.3 %. Specificity varied between 86.0 % and 100.0 %. Sensitivity was higher in IgG assays 410 compared to IgM and IgA, and IgA levels were only negatively correlated with time from 411 symptom onset in one assay. Based on the currently available data, IgM and IgG 412 seroconversion occurs within 10 to 12 days and 12 to 14 days, respectively, after onset of 413 symptoms (20, 33-37). A combination of IgA and IgM assays in SARS-CoV-2 diagnostics may 414 increase sensitivity compared to a single IgG assay at the cost of a lower specificity depending 415 . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 8, 2021. ; https://doi.org/10.1101/2021.02.07.21251062 doi: medRxiv preprint on the assay. ELISAs were more sensitive compared to the lateral flow assays with lower 416 sensitivity values for anti-SARS-CoV-2 IgG in our cohort compared to previously published 417 data (14). Only the SERION ELISA agile SARS-CoV-2 IgG showed a satisfying sensitivity and 418 specificity of more than 98 %. This is in line with data from the only published study on this 419 assay (19). The bead-based surrogate test developed for unequivocally predicting neutralizing 420 activity showed very high specificity and was the only surrogate test which reliably identified 421 COVID-19 samples lacking neutralizing activity against the spike protein. The symptoms most commonly described in this study were olfactory or gustatory dysfunction 423 which occurred in 69 % of the participants. These data are in line with previous studies on this 424 topic (38,39). 425 Limitations of the study are as follows: The severity of disease was not queried and one 426 participant suffered from an immunodeficiency. Recruiting participants in a health care setting 427 resulted in a gender imbalance towards the female gender. Though there are inter-assay 428 differences in sensitivity, the number of participants was not high enough to assess a 429 significant superiority. It is noteworthy that the three immune assays incorporated in this study 430 are based on different antigen components. Whereas the Euroimmun assay is based on S1 431 antigen, Mikrogen recomWell is based on the nucleocapsid protein as antigen and SERION 432 ELISA agile is coated with both S1 and S2 and the nucleocapsid. In the patient, followed over 433 time IgG dynamics differ between the different assays. 434 Data from participants excluded from the main analysis whose COVID-19 disease was not 435 confirmed by PCR, but by an unknown antibody assay, and who did not show neutralizing 436 antibodies highlights the danger of COVID-19 diagnoses by antibody detection with apparently 437 insufficient assays. 438 In this study only one assay (SERION ELISA agile SARS-CoV-2 IgG) showed a sensitivity and 439 specificity of greater than 98 % and may be suitable for widespread assessment whether sera 440 have neutralizing activity against SARS-CoV-2, for the clarification of doubtful PCR results and 441 . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted February 8, 2021. ; https://doi.org/10.1101/2021.02.07.21251062 doi: medRxiv preprint