Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Accuracy of four lateral flow immunoassays for anti SARS-CoV-2 antibodies: a head-to-head comparative study

View ORCID ProfileHayley E Jones, View ORCID ProfileRanya Mulchandani, Sian Taylor-Phillips, A E Ades, Justin Shute, Keith R Perry, Nastassya L Chandra, Tim Brooks, Andre Charlett, Matthew Hickman, Isabel Oliver, Stephen Kaptoge, John Danesh, Emanuele Di Angelantonio, COMPARE study investigators, EDSAB-HOME investigators, View ORCID ProfileDavid Wyllie
doi: https://doi.org/10.1101/2021.01.30.21250777
Hayley E Jones
1Population Health Sciences, Bristol Medical School, University of Bristol, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Hayley E Jones
  • For correspondence: hayley.jones@bristol.ac.uk
Ranya Mulchandani
2Public Health England, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Ranya Mulchandani
Sian Taylor-Phillips
3University of Warwick, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
A E Ades
1Population Health Sciences, Bristol Medical School, University of Bristol, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Justin Shute
2Public Health England, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Keith R Perry
2Public Health England, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nastassya L Chandra
2Public Health England, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tim Brooks
2Public Health England, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andre Charlett
2Public Health England, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Matthew Hickman
1Population Health Sciences, Bristol Medical School, University of Bristol, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Isabel Oliver
2Public Health England, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Stephen Kaptoge
4University of Cambridge, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
John Danesh
4University of Cambridge, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Emanuele Di Angelantonio
4University of Cambridge, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
David Wyllie
2Public Health England, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for David Wyllie
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

Abstract

Background SARS-CoV-2 antibody tests are used for population surveillance and might have a future role in individual risk assessment. Lateral flow immunoassays (LFIAs) can deliver results rapidly and at scale, but have widely varying accuracy.

Methods In a laboratory setting, we performed head-to-head comparisons of four LFIAs: the Rapid Test Consortium’s AbC-19™ Rapid Test, OrientGene COVID IgG/IgM Rapid Test Cassette, SureScreen COVID-19 Rapid Test Cassette, and Biomerica COVID-19 IgG/IgM Rapid Test. We analysed blood samples from 2,847 key workers and 1,995 pre-pandemic blood donors with all four devices.

Findings We observed a clear trade-off between sensitivity and specificity: the IgG band of the SureScreen device and the AbC-19™ device had higher specificities but OrientGene and Biomerica higher sensitivities. Based on analysis of pre-pandemic samples, SureScreen IgG band had the highest specificity (98.9%, 95% confidence interval 98.3 to 99.3%), which translated to the highest positive predictive value across any pre-test probability: for example, 95.1% (95%CI 92.6, 96.8%) at 20% pre-test probability. All four devices showed higher sensitivity at higher antibody concentrations (“spectrum effects”), but the extent of this varied by device.

Interpretation The estimates of sensitivity and specificity can be used to adjust for test error rates when using these devices to estimate the prevalence of antibody. If tests were used to determine whether an individual has SARS-CoV-2 antibodies, in an example scenario in which 20% of individuals have antibodies we estimate around 5% of positive results on the most specific device would be false positives.

Funding Public Health England.

Evidence before this study We searched for evidence on the accuracy of the four devices compared in this study: OrientGene COVID IgG/IgM Rapid Test Cassette, SureScreen COVID-19™ Rapid Test Cassette, Biomerica COVID-19 IgG/IgM Rapid Test and the UK Rapid Test Consortium’s AbC-19™ Rapid Test. We searched Ovid MEDLINE (In-Process & Other Non-Indexed Citations and Daily), PubMed, MedRxiv/BioRxiv and Google Scholar from January 2020 to 16th January 2021. Search terms included device names AND ((SARS-CoV-2) OR (covid)). Of 303 records assessed, data were extracted from 24 studies: 18 reporting on the accuracy of the OrientGene device, 7 SureScreen, 2 AbC-19™ and 1 Biomerica. Only three studies compared the accuracy of two or more of the four devices. With the exception of our previous report on the accuracy of the AbC-19™ device, which the current manuscript builds upon, sample size ranged from 7 to 684. For details, see Supplementary Materials.

The largest study compared OrientGene, SureScreen and Biomerica. SureScreen was estimated to have the highest specificity (99.8%, 95% CI 98.9 to 100%) and OrientGene the highest sensitivity (92.6%), but with uncertainty about the latter result due to small sample sizes. The other two comparative studies were small (n = 65, n = 67) and therefore provide very uncertain results.

We previously observed spectrum effects for the AbC-19™ device, such that sensitivity is upwardly biased if estimated only from PCR-confirmed cases. The vast majority of previous studies estimated sensitivity in this way.

Added value of this study We performed a large scale (n = 4,842), head-to-head laboratory-based evaluation and comparison of four lateral flow devices, which were selected for evaluation by the UK Department of Health and Social Care’s New Tests Advisory Group, on the basis of a survey of test and performance data available. We evaluated the performance of diagnosis based on both IgG and IgM bands, and the IgG band alone. We found a clear trade-off between sensitivity and specificity across devices, with the SureScreen and AbC-19™ devices being more specific and OrientGene and Biomerica more sensitive. Based on analysis of 1,995 pre-pandemic blood samples, we are 99% confident that SureScreen (IgG band reading) has the highest specificity of the four devices (98.9%, 95% CI 98.3, 99.3%).

We found evidence that all four devices have reduced sensitivity at lower antibody indices, i.e. spectrum effects. However, the extent of this varies by device and appears to be less for other devices than for AbC-19.

Our estimates of sensitivity and specificity are likely to be higher than would be observed in real use of these devices, as they were based on majority readings of three trained laboratory personnel.

Implications of all the available evidence When used in epidemiological studies of antibody prevalence, the estimates of sensitivity and specificity provided in this study can be used to adjust for test errors. Increased precision in error rates will translate to increased precision in seroprevalence estimates. If lateral flow devices were used for individual risk assessment, devices with maximum specificity would be preferable. However, if, for an example, 20% of the tested population had antibodies, we estimate that around 1 in 20 positive results on the most specific device would be incorrect.

Competing Interest Statement

JS and KP report financial activities on behalf of WHO in 2018 and 2019 in evaluation of several other rapid test kits. MH declares unrelated and unrestricted speaker fees and travel expenses in last 3 years from MSD and Gillead. JD has received grants from Merck, Novartis, Pfizer and AstraZeneca and personal fees and non-financial support from Pfizer Population Research Advisory Panel. Outside of this work, RB and EL perform meningococcal contract research on behalf of PHE for GSK, Pfizer and Sanofi Pasteur. All other authors declare no conflicts of interest.

Clinical Trial

ISRCTN56609224

Clinical Protocols

http://www.isrctn.com/ISRCTN56609224

Funding Statement

The study was commissioned by the UK Government Department of Health and Social Care, and was funded and implemented by Public Health England, supported by the NIHR Clinical Research Network (CRN) Portfolio. HEJ, AEA, MH and IO acknowledge support from the NIHR Health Protection Research Unit in Behavioural Science and Evaluation at the University of Bristol. DW acknowledges support from the NIHR Health Protection Research Unit in Genomics and Data Enabling at the University of Warwick. STP is supported by an NIHR Career Development Fellowship (CDF-2016-09-018). Participants in the COMPARE study were recruited with the active collaboration of NHS Blood and Transplant (NHSBT) England (www.nhsbt.nhs.uk). Funding for COMPARE was provided by NHSBT and the NIHR Blood and Transplant Research Unit (BTRU) in Donor Health and Genomics (NIHR BTRU-2014-10024). DNA extraction and genotyping were co-funded by the NIHR BTRU and the NIHR BioResource (http://bioresource.nihr.ac.uk). The academic coordinating centre for COMPARE was supported by core funding from: NIHR BTRU, UK Medical Research Council (MR/L003120/1), British Heart Foundation (RG/13/13/30194; RG/18/13/33946) and the NIHR Cambridge Biomedical Research Centre (BRC). COMPARE was also supported by Health Data Research UK, which is funded by the UK Medical Research Council, Engineering and Physical Sciences Research Council, Economic and Social Research Council, Department of Health and Social Care (England), Chief Scientist Office of the Scottish Government Health and Social Care Directorates, Health and Social Care Research and Development Division (Welsh Division), Public Health Agency (Northern Ireland), British Heart Foundation, and Wellcome. JD holds a British Heart Foundation professorship and an NIHR senior investigator award. SK is funded by a BHF Chair award (CH/12/2/29428). The views expressed are those of the authors and not necessarily those of the NHS, NIHR or Department of Health and Social Care.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The study was approved by NHS Research Ethics Committee (Health Research Authority, IRAS 284980) on 02/06/2020 and PHE Research Ethics and Governance Group (REGG, NR0198) on 21/05/2020.

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Data Availability

Data are available on reasonable request to the corresponding author.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission.
Back to top
PreviousNext
Posted February 01, 2021.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Accuracy of four lateral flow immunoassays for anti SARS-CoV-2 antibodies: a head-to-head comparative study
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Accuracy of four lateral flow immunoassays for anti SARS-CoV-2 antibodies: a head-to-head comparative study
Hayley E Jones, Ranya Mulchandani, Sian Taylor-Phillips, A E Ades, Justin Shute, Keith R Perry, Nastassya L Chandra, Tim Brooks, Andre Charlett, Matthew Hickman, Isabel Oliver, Stephen Kaptoge, John Danesh, Emanuele Di Angelantonio, COMPARE study investigators, EDSAB-HOME investigators, David Wyllie
medRxiv 2021.01.30.21250777; doi: https://doi.org/10.1101/2021.01.30.21250777
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Accuracy of four lateral flow immunoassays for anti SARS-CoV-2 antibodies: a head-to-head comparative study
Hayley E Jones, Ranya Mulchandani, Sian Taylor-Phillips, A E Ades, Justin Shute, Keith R Perry, Nastassya L Chandra, Tim Brooks, Andre Charlett, Matthew Hickman, Isabel Oliver, Stephen Kaptoge, John Danesh, Emanuele Di Angelantonio, COMPARE study investigators, EDSAB-HOME investigators, David Wyllie
medRxiv 2021.01.30.21250777; doi: https://doi.org/10.1101/2021.01.30.21250777

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Epidemiology
Subject Areas
All Articles
  • Addiction Medicine (270)
  • Allergy and Immunology (552)
  • Anesthesia (135)
  • Cardiovascular Medicine (1757)
  • Dentistry and Oral Medicine (238)
  • Dermatology (173)
  • Emergency Medicine (312)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (658)
  • Epidemiology (10796)
  • Forensic Medicine (8)
  • Gastroenterology (589)
  • Genetic and Genomic Medicine (2946)
  • Geriatric Medicine (287)
  • Health Economics (534)
  • Health Informatics (1927)
  • Health Policy (836)
  • Health Systems and Quality Improvement (745)
  • Hematology (291)
  • HIV/AIDS (630)
  • Infectious Diseases (except HIV/AIDS) (12515)
  • Intensive Care and Critical Care Medicine (688)
  • Medical Education (299)
  • Medical Ethics (86)
  • Nephrology (324)
  • Neurology (2798)
  • Nursing (151)
  • Nutrition (433)
  • Obstetrics and Gynecology (559)
  • Occupational and Environmental Health (597)
  • Oncology (1463)
  • Ophthalmology (444)
  • Orthopedics (172)
  • Otolaryngology (255)
  • Pain Medicine (190)
  • Palliative Medicine (56)
  • Pathology (380)
  • Pediatrics (866)
  • Pharmacology and Therapeutics (366)
  • Primary Care Research (337)
  • Psychiatry and Clinical Psychology (2639)
  • Public and Global Health (5363)
  • Radiology and Imaging (1011)
  • Rehabilitation Medicine and Physical Therapy (595)
  • Respiratory Medicine (726)
  • Rheumatology (330)
  • Sexual and Reproductive Health (289)
  • Sports Medicine (279)
  • Surgery (327)
  • Toxicology (47)
  • Transplantation (150)
  • Urology (125)