Abstract
With the availability of vaccines, commercial assays detecting anti-SARS-CoV-2 antibodies (Ab) evolved towards quantitative assays directed to the spike glycoprotein or its receptor binding domain (RBD). The main objective of the present study was to compare the Ab titers obtained with quantitative commercial binding Ab assays, after 1 dose (convalescent individuals) or 2 doses (naive individuals) of vaccine, in healthcare workers (HCW).
Antibody titers were measured in 263 sera (from 150 HCW) with 5 quantitative immunoassays (Abbott RBD IgG II quant, bioMerieux RBD IgG, DiaSorin Trimeric spike IgG, Siemens Healthineers RBD IgG, Wantai RBD IgG). One qualitative total antibody anti RBD detection assay (Wantai) was used to detect previous infection before vaccination. The results are presented in binding Ab units (BAU)/mL after application, when possible, of a conversion factor provided by the manufacturers and established from a World Health Organization (WHO) internal standard.
There was a 100% seroconversion with all assays evaluated after two doses of vaccine. With assays allowing BAU/ml correction, Ab titers were correlated (ρ= 0.84-0.99). However, a significant difference between values persisted. The titer differences varied by a mean 3.04% between Siemens and bioMerieux assays to 50.54% between Siemens and DiaSorin assays.
Titer harmonization is still to be improved despite better results were obtained between assays detecting the same Ab against the same antigen. The next step towards a true standardization of the assays would be to include the International Standard in the calibration of each assays to express the results in IU/mL.
Introduction
Since the end of 2020, severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) vaccines have been developed and are now available worldwide with the aim of achieving herd immunity to control the pandemic. Vaccine immunity involves both cellular and humoral pathways. Cellular immunity is not easy to assess on a large scale, as is the neutralizing humoral response owing to requirement for a biosafety level 3 (BSL3) containment laboratory; the evaluation of vaccine effectiveness therefore mainly relies on high throughput serological tests to assess individual humoral immunity as well as monitoring SARS-CoV-2 seroprevalence (1).
To consider binding antibodies (Ab) as a marker of vaccine effectiveness, several conditions must be met. First, binding Ab assays should be quantitative; second, titers should be consistent between different assays; third, binding Ab titers should correlate with neutralizing Ab titers; fourth, the minimum binding Ab titer associated with virus neutralization must be found; and fifth, the association between neutralizing Ab and vaccine protection must be demonstrated. It can be considered that the first and fifth conditions have been met given that commercial tests for the quantitative detection of binding Ab have been developed (2–7), and that the role of neutralizing Ab in the infection protection have been demonstrated in animals and humans (8–11); this is not the case for the other conditions. In particular, the second point is of importance for widespread evaluation of vaccines, but until now Ab titers were often expressed as an index or unit with regard to an internal standard that differs between manufacturers. Recently, the World Health Organization (WHO) has developed an international standard (12) against which each supplier can standardize their assay, allowing comparability of titers between kits. The present study was conducted to evaluate the performance of commercial antibody assays in detecting vaccination-associated anti-SARS-CoV-2 Ab seroconversion; the main objective was to compare Ab titers from quantitative assays after conversion of titers using the WHO standard.
Materials and Methods
Antibody binding assays
Six Ab binding assays were investigated according to the protocol recommended by each manufacturer (the characteristics of the assays are summarized in Table 1). Five were quantitative: Siemens Healthineers (Erlangen, Germany) Atellica® IM SARS-CoV-2 IgG (sCOVG), DiaSorin (Saluggia, Italy) Liaison® SARS-CoV-2 TrimericS IgG, bioMérieux (Marcy l’Etoile, France) Vidas® SARS-CoV-2 IgG, Abbott (Abbott park, Il, USA) Architect SARS-CoV2 IgG II Quant, and Wantai (Beijing, China) SARS-CoV-2 IgG assays. The Wantai SARS-CoV-2 total antibody assay is qualitative and was selected to detect a previous infection before vaccination based on its better sensitivity previously shown in infected individuals (13). The First International Standard developed by the WHO (National Institute for Biological Standards and Control code: 20/136) corresponds to lyophilized pooled plasma from patients who had been infected with SARS-CoV-2; after reconstitution, the solution contains 1000 binding antibody units (BAU) per mL (12). For conversion of titers obtained using the quantitative assays, the concentrations expressed in arbitrary units per mL, or index according to the assay, (Table 1) were converted to BAU/mL using the conversion factors provided by the manufacturer (with the exception of the Wantai SARS-CoV-2 IgG assay for which the conversion factor was not available and presented here only to compare the positivity rate between assays); these were 21.8 for the Siemens assay, 2.6 for the DiaSorin assay, 20.33 for the bioMérieux assay, and 0.142 for the Abbott assay (considering that 1 BAU/mL = conversion factor x AU/mL or index). Samples with results above the upper limit of quantification were tested again after dilution (1/5 when above 3270 BAU/mL for the Siemens assay, 1/20 when above 2080 BAU/mL for the DiaSorin assay, and 1/20 when above 18 index for the bioMérieux assay).
Samples from the study population
A prospective longitudinal cohort study was conducted at the laboratory associated with the national reference center for respiratory viruses (University Hospital of Lyon, France). Healthcare workers, excluding pregnant women (HCW; n=150) who were scheduled to receive 2 doses of Pfizer BioNtech vaccine (n=94; BNT162b2/BNT162b2) or 1 dose of AstraZeneca vaccine followed by 1 dose of Pfizer BioNtech vaccine (n=56; ChAdOx1/BNT162b2) were included. Blood samples were collected i) before the first dose of vaccine, ii) before the second injection of vaccine corresponding to 4 weeks after the first dose for participants vaccinated with 2 doses of Pfizer BioNtech vaccine or 12 weeks for those vaccinated with the AstraZeneca vaccine, and iii) 4 weeks after the full vaccination. The pre-vaccination blood sample was only used to document a previous SARS-CoV-2 infection. Among the participants 26 who were previously infected with SARS-CoV-2 (convalescent group; 17.4%) had only one vaccine injection (Pfizer BioNtech, n=15 or Astra Zeneca, n=11); for these the second sample was omitted. Three participants were infected with SARS-CoV-2 between the 2 doses. Sera were immediately stored at −80°C after blood sampling.
Written informed consent was obtained from all participants; ethics approval was obtained from the regional review board for biomedical research in April 2020 (Comité de Protection des Personnes Sud Méditerranée I, Marseille, France; ID RCB 2020-A00932-37), and the study was registered on ClinicalTrials.gov (NCT04341142).
Samples for specificity assessment
Specificity was assessed from 39 pre-pandemic sera specimens, collected between 2017 and 2019. These pre-pandemic sera were positive for antinuclear Ab (n=10), rheumatoid factor (n=10), autoAb found in patients suffering from Crohn’s disease (n=10), or for anti-mycoplasma Ab (IgM; n=9). The 39 pre-pandemic sera specimens were tested with the Wantai total Ab and bioMérieux IgG assays; due to insufficient volume, only 20 of them could be tested with the Siemens and DiaSorin IgG assays.
Statistical analyses
Results were expressed by the median and interquartile range [IQR]. Paired comparison between assays was performed using the Wilcoxon test. The correlation between concentrations obtained by each assay was investigated using Pearson correlation coefficients and 95% confidence interval (CI). The Bland-Altman method was used to measure the mean difference and 95% limit of agreement between log-transformed concentrations obtained with each assay Statistical analyses were conducted using GraphPad Prism® software (version 8; GraphPad software, La Jolla, CA, USA). A p-value <0.05 was considered statistically significant.
Results
In the first part of the study, the performances of the 6 assays were compared to verify whether the ability to detect anti-SARS-CoV-2 antibodies of the Wantai total Ab assay, which is known to be the most sensitive post infection (13), was similar after vaccination, and whether the sensitivity of qualitative and quantitative assays were also similar. The sera collected from patients scheduled to receive only Pfizer BioNtech vaccine (n=94) were used for this evaluation. Four weeks after the first injection, the proportion of positive samples was over 90% for all assays except the Wantai assay detecting total antibodies (86.1%). Four weeks after the second injection anti SARS CoV-2 antibodies were detected for all participants with all the assays (Table 1). All assays were negative on pre-pandemic samples from patients with autoantibodies or mycoplasma infection (100% specificity).
The second part of this study was to compare Ab titers after BAU/mL conversion; for this, only the assays adapted or developed for the quantification of Ab induced by vaccine were tested. The Siemens, DiaSorin and bioMérieux assays were compared using sera samples collected before the second injection of vaccine and those collected after full vaccination, and for which assays gave a positive quantitative result (263 samples). The median [IQR] values obtained were 708.3 [90.47; 2439] BAU/mL for the Siemens assay, 1220 [241.8; 3370] BAU/mL for DiaSorin, and 906.7 [128.1; 2289] BAU/mL for bioMérieux assays; there was a significant difference in titers between DiaSorin and Siemens (p<0.0001), as well as DiaSorin and bioMérieux (p<0.0001) assays. The difference in median titers between Siemens and bioMérieux assays was not significant (Figure 1A). There was a strong correlation between assays; the Pearson correlation coefficient (ρ [95%CI]) was 0.94 [0.92; 0.95] between DiaSorin and Siemens assays, 0.86 [0.82; 0.89] between DiaSorin and bioMérieux assays, and 0.84 [0.80; 0.87] between Siemens and bioMérieux assays (Figure 2A, B, C).
According to the Bland-Altman method (Figure 3) there was a mean difference in titers expressed as BAU concentrations of 3.04% [-72.37%; 78.38%] between the Siemens and bioMérieux assays, 47.39% [-127.36%; 32.57%] between DiaSorin and bioMérieux, and 50.54% [-27.26%; 128.33%] between DiaSorin and Siemens assays.
There were 77 samples from HCW scheduled to receive only Pfizer BioNtech vaccine (68 after the first dose and 9 after the second dose) tested using the Abbott assay. The median [IQR] values obtained were 114.5 [55.38; 382.2] BAU/mL for the Siemens assay, 116.3 [51.19; 295.7] for the Abbott assay, 169.1 [80.71; 384.5] for the bioMérieux assay, and 325 [131.8; 811.2] for the DiaSorin assay. The titers were significantly different between Abbott and each other assay (p<0.0001; Figure 1B), but the data from these assays were strongly correlated; the Pearson correlation coefficient (ρ [95%CI]) was 0.99 [0.98; 0.99] between Abbott and Siemens assays, 0.98 [0.96; 0.98] between Abbott and DiaSorin assays, and 0.99 [0.99; 0.997] between Abbott and bioMérieux assays (Figure 2D, E, F).
Discussion
In the present cohort of vaccinated HCW, the performance of first generation serological assays, developed for diagnosis purpose, as well as second generation ones, adapted or developed in order to assess the vaccine humoral response, were similar. However, first generation assays are not useful to monitor vaccination effectiveness since they are not quantitative or could not be compared to others. The more important finding of the present study was that the quantitative assays, whose results could be standardized to BAU/ml, produce results that are correlated to each other. The Ab titers obtained with the DiaSorin assay, although correlated with those of the others assays, remained higher after conversion using the WHO standard. This may be related to difference in the antigens targeted, as the DiaSorin assay is the only one evaluated in this study, to use the trimeric spike protein (2). The WHO standard was obtained from the plasma of convalescent patients (12) and must therefore contain antibodies against numerous epitopes, thus the DiaSorin assay is probably capable of reacting with more antibodies than assays detecting only antibodies specific for RBD.
There are currently very few reports that have examined Ab binding assays with the use of the WHO standard (3, 7). Perkman et al. (7) compared 4 assays detecting binding Ab, including the Liaison anti SARS-CoV-2 trimericS IgG from DiaSorin and the Architect anti SARS-CoV-2 RBD IgG from Abbott, used in the present study. In vaccinated individuals, after the first dose of vaccine, titers varied significantly between the assays, but they indicated that the recalculation in BAU/ml with the conversion factor given by the manufacturer did not solve error problems between tests. However, the assays compared were more different in their format than those investigated herein: total anti-RBD Ig versus anti-RBD IgG, anti-monomeric spike IgG or anti-trimeric spike IgG. Interestingly, comparison of Abbott and DiaSorin assays found, as was the case herein, higher titers for the DiaSorin assay. Later, Bradley et al. (3) performed linear regressions from sample dilutions of the WHO standard to determine a detection limit in international units (IU) per ml for each test; the authors confirmed the linearity of the Abbott anti-SARS-CoV-2 IgG II Quant assay over the analytical measurement interval but the conversion factor found seemed to be higher than indicated by the manufacturer (1 IU/ml gave 6,1 arbitrary units/ml while Abbott indicated that 1 IU correspond to 7.1 arbitrary units). Taken together, these data suggest that there are remaining differences between assays after conversion in BAU/ml and that this could be due to the incorrect adjustment of the correction factor by the manufacturers. The next step for a true harmonization would therefore be to use the international standard to calibrate each assay instead of applying a conversion factor to a result obtained with an assay previously calibrated with an internal standard (14).
A limitation of this study is that the specificity analysis was limited to a small group of patients with auto-immune diseases or mycoplasma infection. However, assay specificity analyses have been performed by manufacturers and independent groups (2, 4–6) showing specificity ≥99% for all the quantitative assays. In addition, not all commercial quantitative anti-SARS-CoV-2 Ab assays were evaluated limiting the scope of the conclusions. Furthermore, neutralizing Ab were not investigated that could have helped determine whether anti-RBD or anti-spike assays are the most correlated with virus neutralization. However, before investigating this, harmonization of neutralizing Ab titers is also necessary to determine a common threshold from which vaccine protection could be predicted, allowing then to find the corresponding threshold with high throughput binding Ab assays. A study comparing different cell-based assays (with either live or pseudotyped viruses) to measure neutralization in vitro is rather reassuring, although differences were found according to the viruses used for pseudotyping (15). However, comparison of cell-based assays with surrogate virus neutralization tests (sVNT) that are based on ELISA, and measuring the competition of Ab and RBD for the binding to ACE, the cellular entry receptor of the virus, did not find good agreement; this is inconvenient, as these assays could be promising given that they have potential for large-scale.
In conclusion, the evaluated assays correlated well with each other but a difference in titers remained after adjustment to the same International Standard. Thus, the titer harmonization is not yet completely achieved, but it is better between assays detecting the same Ab against the same antigen than between assays with different targets.
Data Availability
The data that support the findings of this study are available from the corresponding author, STA, upon reasonable request.
Acknowledgements
This study was supported by Hospices Civils de Lyon and Fondation des Hospices Civils de Lyon. The respective suppliers kindly provided all the serological kits used in this study.
We thank all the staff members of the occupational health and medicine department of the Hospices Civils de Lyon who contributed to the sample collection. We thank all Clinical Research Associates, for their excellent work. We thank Karima Brahima and all the members of the clinical research and innovation department for their reactivity (DRCI, Hospices Civils de Lyon). Human biological samples and associated data were obtained from NeuroBioTec (CRB HCL, Lyon France, Biobank BB-0033-00046). We thank all the technicians from the virology laboratory of the Hospices Civils de Lyon who performed the assays on the automated platforms. Lastly, we thank all the healthcare workers for their participation in this clinical study, and Philip Robinson for critical reading of the manuscript and language correction.