Diagnostic accuracy of rapid point-of-care tests for diagnosis of current SARS-CoV-2 infections in children: A systematic review and meta-analysis

Eligibility criteria

We included diagnostic cross-sectional and cohort studies that evaluated the clinical performance of rapid point-of-care tests for detecting current SARS-CoV-2 infections against the reference standard in paediatric or mixed-age populations. Assessing analytical performance parameters such as the analytical sensitivity (limit of detection) or the analytical specificity (cross-reactivity) was not subject of the current review. Diagnostic case-control studies were excluded as they reflect the test’s performance under ideal conditions and, therefore, often overestimate the diagnostic accuracy [16]. Moreover, studies evaluating serological tests were excluded as such tests are not suitable for the initial diagnosis of current SARS-CoV-2 infection [17]. We considered a study as eligible if the study population comprised at least ten paediatric study participants, each identified as positive or negative by the reference standard. In the absence of a true gold standard, laboratory-based RT-PCR alone or in combination with clinical findings or clinical follow-up was defined as the reference standard since it reflects the best available method for diagnosing individuals currently infected with SARS-CoV-2 [7]. Further, we required reporting of data that allowed constructing a complete 2×2 contingency table. The full set of eligibility criteria is shown in Table 1.

For studies that did not fully meet the inclusion criteria for the population, index test and/or reference standard, we required that at least 80% of the paediatric (sub-)population matched the population we defined for this systematic review. Studies were excluded, if the index test and the reference standard were performed in less than 80% of the paediatric study population. Besides journal articles, reports (including clinical study reports) that adhered to reporting standards such as STARD (Standards for Reporting of Diagnostic Accuracy Studies) [18] or recommendations given by government agencies [10,19] were considered eligible for inclusion. Studies that mentioned the inclusion of paediatric study participants without reporting any corresponding outcome data but otherwise met the eligibility criteria were preliminary included, and study authors were contacted and asked to provide such data. Further, if the study population’s baseline characteristics included information on age, we estimated the proportion of paediatric study participants assuming ages of study participants following a normal distribution and the proportion of PCR-positive paediatric assuming no changes in the PCR positivity rate among age groups. We contacted authors if we estimated at least 10 PCR-positive paediatric study participants in the study population.

Information sources

We performed a comprehensive search for primary studies and secondary publications (systematic reviews and Health Technology Assessment (HTA) reports) in the following electronic bibliographic databases: MEDLINE (Ovid), Embase (Ovid), the Cochrane Library (Wiley), and preprint servers (Europe PMC) including medRxiv and bioRxiv (see [20] for full list of included preprint servers). Here, secondary publications were solely used as sources for potentially relevant studies. In addition, we searched two study registries (ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform (ICTRP)) for relevant clinical studies. Other information sources comprised the International HTA Database, the Foundation of Innovative Diagnostics (FIND) COVID-19 website, and the Evidence Search and Guidance websites of Britain’s National Institute for Health and Care Excellence (NICE).

Search strategy

In accordance with the Cochrane Handbook for DTA Reviews [21], the search strategy included concepts addressing the index test and the target condition. The development of the search strategy followed an objective approach that involved text-analytic procedures to identify candidate search terms based on the method described by Hausner et al. [22]. One researcher performed analyses of simple word frequencies and keywords-in-contexts in R using the “quanteda” package [23]. Because of substantial differences between types of tests, separate test sets were used to identify candidate search terms for antigen tests and molecular tests, respectively. Test sets included potentially relevant studies (irrespective of paediatric study participants) from the Cochrane Review by Dinnes et al. [8] and from a frequently updated website that lists DTA studies on antigen tests [24]. Due to a limited number of potentially relevant references addressing rapid molecular tests for point-of-care usage, the draft search strategy was supplemented by search terms derived from a conceptual approach. Furthermore, brand names of tests included in the Cochrane Review were added to increase sensitivity. The final search strategy was tested for completeness against the validation sets and relevant references of studies that included paediatric participants identified via exploratory searches beforehand. Prior to execution, the search strategy was peer-reviewed by a senior information specialist following the Peer Review of Electronic Search Strategies (PRESS) Guideline statement [25].

We only searched for publications published after December 2019, as we were only interested in literature published after the emergence of SARS-CoV-2. Further, we limited our search to publications written in English or German. Since Embase and MEDLINE provided comprehensive search filters for SARS-CoV-2 related literature via Ovid, our concept addressing the target condition was not used in these two searches.

To acknowledge the unprecedented role of preprints in the rapid dissemination of SARS-CoV-2 related research, we also searched for relevant preprints. Due to the direct availability of full texts of preprints and to increase the efficiency of the information retrieval, a further concept addressing the target population was defined and used in addition to the standard search strategy for identifying potentially relevant preprints directly at the full-text level. We assumed that this approach allowed to increase the precision of the overall search without a relevant reduction of its comprehensiveness.

All search strategies are provided in Appendix 1. The last search in bibliographic databases and study registries was conducted on May 7, 2021. Other information sources were last searched on May 24, 2021. Endnote X9.3 was used for citation management. Due to the more specific separate search for preprints at full-text level, any preprint records identified from MEDLINE were removed. Duplicates were initially eliminated via Ovid’s deduplication feature. After exporting all identified references from Ovid, duplicates were identified in R by comparing digital object identifiers (DOIs) of references from MEDLINE and Embase, and the Embase records of duplicates were removed. Remaining duplicates were manually removed in EndNote X9.3 and by using Endnote’s “find duplicates” function. Further, records from ClinicalTrials.gov that were retrieved from the WHO’s ICTRP website were removed since directly accessing ClinicalTrials.gov’s registry data allows for a more comprehensive search for relevant studies.

Study selection

The screening of literature retrieved from bibliographical databases involved a two-step screening procedure and was performed independently by two researchers using the web-based Trial Selection Database (webTSDB) [26]. In a first step, potentially eligible primary studies and secondary publications were identified from screening titles and abstracts of retrieved citations. In a second step, the full texts of these articles were obtained and evaluated. Publications that met the eligibility criteria were included. Any discrepancies were resolved by consensus between the two researchers before finalizing each screening step. Reference lists of relevant systematic reviews and HTAs (independent of mentioning paediatric study participants) were manually screened to identify further relevant studies. For the screening of records from study registries, both screening steps were combined. Furthermore, documents identified through searching other information sources were screened for eligibility or information about potentially relevant studies.

Data collection

The individual steps of data collection were performed by one researcher. All output was checked by a second researcher to ensure its validity and completeness. Any disagreements were resolved by consensus. At first, a standardized Excel spreadsheet was developed for data extraction. The spreadsheet was piloted before data extraction commenced. Extracted data included information on the general study characteristics, study participant characteristics, index test, reference standard, flow and timing, and reported outcomes. A complete list of data extraction items is presented in Table 2.

Quality assessment

We used the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool [27] to evaluate the methodological quality and applicability of the included studies at the study level. The tool requires assessing the risk of bias in the four domains patient selection, index test, reference standard, and flow and timing. Further, applicability concerns need to be assessed for the first three domains. Judgments are facilitated by answering pre-defined signalling questions with “yes”, “no”, or “unclear” for each domain. The tool was tailored to our review by adding one signalling question and review-specific guidance was provided to facilitate judgments, see Appendix 2. The quality assessment of each included study was performed by one researcher. A second researcher verified all judgments. Any disagreements were resolved by consensus. The results were summarised in the text and visualized as table and figure.

Diagnostic accuracy measures and data synthesis

For each included study, diagnostic sensitivity, diagnostic specificity, positive predictive value (PPV) and negative predictive value (NPV) with corresponding 95% confidence intervals (CIs) were calculated based on the extracted 2×2 tables. Individual study participants were used as the unit of analysis throughout this work. If a study reported repeat testing of individuals only the initial test was included in our analyses. If a study evaluated more than one test in the same study population, we reported all test evaluations, but only one randomly chosen test was included in the meta-analyses to avoid the necessity to adjust for multiplicity.

The meta-analyses were based on recommendations provided in the methodological guideline “Meta-analysis of diagnostic test accuracy studies” by the European Network for Health Technology Assessment (EUnetHTA) [28]. Summary estimates for sensitivity and specificity were derived as follows: if sufficient data was available and the level of heterogeneity allowed meaningful statistical pooling, bivariate meta-analysis with random effects following the approach by Reitsma et al [29–31] was performed. Otherwise, separate univariate meta-analysis was performed. The bivariate approach required a continuity correction to handle zero cells in 2×2 tables. Thus, in studies where zero events were observed in one of the four cells, a continuity correction was applied by adding 0.5 to all four cells.

Depending on the availability of suitable data, subgroup analyses were performed to assess variables that could have an impact on a test’s diagnostic accuracy, such as the study participants’ presence of symptoms prior to testing and the duration of symptoms prior to testing. The influence of the publication status (preprint vs. peer-reviewed article) was evaluated as well as subgroup analyses with respect to the type of test (antigen vs. molecular; most commonly used antigen tests), setting (community vs. hospital-based), sample type ((oro-) nasopharyngeal vs. anterior nasal for index test and reference standard, respectively), end-user (layperson (self-testing) vs. trained staff/health care worker), and RT-PCR Ct value (cut-off values of 25 and 30). Differences between subgroups were assessed within the bivariate model and tested for statistical significance using the likelihood ratio test between the standard model and the model, which includes the corresponding variable. In the case of few studies in a subgroup analysis, univariate analysis for sensitivity and specificity were performed as sensitivity analysis and results were reported if remarkable differences between bivariate and univariate analysis were observed.

All statistical analyses were performed using the statistical platform R version 4.1.0 [32]. Bivariate meta-analysis was performed, along with the construction of the corresponding figures, with the package “mada” [33], while univariate meta-analysis was performed with the package “meta” [34] and “PropCIs” [35]. 95% CIs were computed using the approach proposed by Wilson [36].

Patient and public involvement

Neither patients nor members of the public were actively involved in the conduct of this systematic review. However, the development of the research question was motivated by public discussions about SARS-CoV-2 antigen testing in children. Further, we were – and still are – in continuous exchange with teachers, parents, and children about practical aspects and the impact of current testing programmes in schools and kindergarten.

Study selection

Overall, 3011 records were retrieved from five bibliographic databases. The PRISMA flow diagram is shown in Figure 1 and outlines the process of identifying relevant studies from different information sources. References that were excluded at the full-text level can be found in Appendix 3 with the reason for their exclusion. After removing 36 preprint records identified via MEDLINE and 680 duplicate records, 2295 records were screened for eligibility. 2078 records were excluded at the title/abstract level. Full-text publications of 217 records were retrieved for further assessment. Nine studies [37–45] met all eligibility criteria for inclusion. Furthermore, 21 studies [46–66] were identified as eligible for author queries to obtain study data on paediatric subgroups. Authors of nine studies did not respond to our request for data [46,47,52,53,55,57–59,62]. In four cases [49,60,65,66], authors reported that the required number of individuals who tested positive or negative by the reference standard was not reached, and in one case [56] no data on age were recorded. Eventually, author queries led to the inclusion of eight further studies [48,50,51,54,61,63,64,67] resulting in a total of 17 relevant studies for this review (12 peer-reviewed journal articles and five preprints). The full list of included studies is reported in Table 3.

• Download figure
• Open in new tab

Figure 1:

PRISMA Flow diagram illustrating the selection process of primary studies included in this systematic review and meta-analysis.

Furthermore, we screened 113 records identified from study registries and 323 records identified from other information sources. The search for studies in study registries allowed to identify four planned or ongoing and four completed studies with no results posted, see Table 4 for further details. Information retrieval from other information sources included screening 18 records retrieved from the FIND website, 78 records from NICE Evidence Search, 28 records from NICE, and 23 records from reference lists of six systematic reviews [8,68–72] identified via searching bibliographic databases. As a result, no additional study that met the inclusion criteria was identified.

Study characteristics

All 17 included studies (6355 paediatric study participants) evaluated the performance of antigen tests against the reference standard RT-PCR. The main study characteristics for each individual study are summarised in Table 5, further details are reported in Table 6 and Table 7. While 14 studies evaluated the test performance in mixed-age populations, including 24 to 928 paediatric study participants, three studies with a sample size between 440 and 1620 individuals exclusively recruited children. In eight studies, the purpose of testing included diagnostic testing of symptomatic individuals suggestive of SARS-CoV-2 infection. Six studies reported the inclusion of asymptomatic individuals at an increased risk of infection due to previous exposure to SARS-CoV-2. Here, “asymptomatic” refers to any individual who is healthy, infected but pre-symptomatic or infected but without symptoms. Evaluating the performance of antigen tests in a screening setting (e.g. community mass testing) was the main objective of six studies. Eight antigen tests (six lateral flow immunochromatographic assays and two fluorescent immunoassays) from six different brands were used in 18 test evaluations, whereas antigen tests by Abbott were most investigated (Panbio COVID-19 Ag Rapid Test n=6, BinaxNOW COVID-19 Ag Card n=5). In more than half of the test evaluations (n=11), nasopharyngeal samples were collected for the index test. Six test evaluations used anterior nasal specimens for the index test. In all studies, the reference standard was RT-PCR performed in a laboratory setting.

Risk of bias and applicability

The results of the quality assessment are summarised in Table 8 and Figure 2. Quality among studies varied. Only one study was at low risk of bias in all four domains of the QUADAS-2 tool. For patient selection, more than half of the studies were at high (n=1) or unclear (n=12) risk of bias because inadequate exclusion of participants occurred, or it was not clear whether a consecutive or random sample was enrolled into the study. All but one study was judged as having an unclear risk of bias for the reference standard due to insufficient reporting of blinding. Risk of bias in the flow and timing domain was high in three studies due to more than 5% of missing outcome data.

• Download figure
• Open in new tab

Figure 2:

QUADAS-2 risk of bias and applicability concerns – graphical summary showing the review authors’ judgment about each domain as percentages across 18 test evaluations reported in 17 included studies.

Overall applicability concerns were high in three studies due to high concerns in either the patient selection or index test domain. Three studies were of low concern and the remaining 11 were rated unclear due to insufficient reporting in at least one domain.

Results of individual studies

RT-PCR positivity rate, diagnostic sensitivity, diagnostic specificity as well as PPV and NPV (and their 95% CIs) of individual studies based on data from 2×2 contingency tables for paediatric populations are reported in Table 9 and Figure 3. The RT-PCR positivity rate, which corresponds to the SARS-CoV-2 prevalence in the sample population, varied between 4.1% and 50%, with a median of 14,5% over n=17 studies. The sensitivity and specificity ranged from 33.3% to 85.7% and 91.7% to 100%, respectively. PPV and NPV ranged from 60.0% to 98.7% and 73.3% to 98.9%, respectively.

• Download figure
• Open in new tab

Figure 3:

Forest plot of sensitivity and specificity of antigen tests in entire paediatric study populations irrespective of symptoms. The point estimates of sensitivity and specificity from each study (identified by name of first author) are shown as squares, the corresponding 95% confidence intervals (95% CIs) are represented as horizontal lines. TP: true positive, FN: false negative, TN: true negative, FP: false positive.

For individual studies, separate analyses for subgroups based on symptom status are reported in Tables 10-12 and Figure 4. Here, populations were defined as symptomatic or asymptomatic if at least 80% of paediatric study participants were reported as being symptomatic or asymptomatic, respectively. Mixed populations refer to populations with no predominant symptom status. RT-PCR positivity rates of the primary analysis population and the different subgroups based on symptom status are presented in Figure 5. Two studies [51,54] were performed in high-prevalence populations with RT-PCR positivity rates of 38.7% and 50.0%, respectively. The median RT-PCR positivity rate was 13.2% (n=10 test evaluations) in asymptomatic populations, 13.8% in mixed populations (n=3 test evaluations), and 25.7% in symptomatic populations (n=13 test evaluations). Thus, we observed a slight trend in the RT-PCR positivity rate with respect to the proportion of symptomatic subjects. Although the bivariate meta-analysis might be influenced by the difference in the prevalence [73], we did not directly analyse the impact of the prevalence within the bivariate model. Instead, we performed a subgroup analysis with respect to symptom status (see below).

• Download figure
• Open in new tab

Figure 4:

Forest plot of sensitivity and specificity of antigen tests in (a) symptomatic, (b) asymptomatic, and (c) mixed paediatric study populations. The point estimates of sensitivity and specificity from each study (identified by name of first author) are shown as squares, the corresponding 95% confidence intervals (95% CIs) are represented as horizontal lines. TP: true positive, FN: false negative, TN: true negative, FP: false positive.

• Download figure
• Open in new tab

Figure 5:

Box and whisker plots of the RT-PCR positivity rates in entire paediatric study populations irrespective of symptoms and in asymptomatic, mixed, and symptomatic paediatric study populations. The box is drawn from the first to the third quartile, defined via hinges. The bold horizontal line in the box denotes the median. The whiskers extend out of the box by 1.5 times the difference between the hinges. Outliers are plotted as small circle. The width of the box corresponds to the number of considered studies.

Synthesis of results

In our primary meta-analyses, we used data from 17 studies evaluating the diagnostic accuracy of antigen tests in paediatric participants. Estimated pooled sensitivity and specificity were 64.2% (95% CI: 57.4% to 70.5%) and 99.1% (95% CI: 98.2% to 99.5%), respectively. While the estimates for the sensitivity revealed high heterogeneity and thus justified the application of the bivariate model with random effects, the estimates for the specificity were limited to a small range, as shown in Figure 6. As a consequence, the estimated summary receiver operating characteristic (SROC) curve cannot be meaningfully interpreted. As pre-specified in the protocol, we performed subgroup analysis evaluating the diagnostic accuracy according to symptom status. Estimated pooled sensitivity and specificity in asymptomatic children was 56.2% (95% CI: 47.6% to 64.4%) and 98.6% (95% CI: 97.3% to 99.3%), respectively, based on data from 2439 asymptomatic children in 10 studies. Estimated pooled sensitivity and specificity in symptomatic children was 71.8% (95% CI: 63.6% to 78.8%) and 98.7% (95% CI: 96.6% to 99.5%), respectively, based on data from 3413 symptomatic children in 13 studies. Estimated pooled sensitivity and specificity in the mixed population of symptomatic and asymptomatic children from three studies including 419 children was 63.4% (95% CI: 37.3% to 83.5%) and 98.7% (95% CI: 90.8% to 99.8%), respectively. The corresponding SROC curves are shown in Figure 7. The likelihood ratio test (LRT) for differences between the three groups revealed a p-value of p_LRT=0.066. Results for the other subgroup analyses did not show relevant differences in the pooled estimates (setting p_LRT=0.400; index test sample type p_LRT=0.303; reference standard sample type p_LRT=0.723; RT-PCR Ct cut-off value p_LRT=0.105; publication status p_LRT=0.551; test type (most used) p_LRT=0.146). Due to insufficient data, we did not perform subgroup analysis with respect to test type (antigen vs. molecular) and end-user (layperson (self-testing) vs. trained staff/health care worker). Except for one study [61,74] where the testing procedure involved (supervised) self-collection of samples by study participants, in all other studies, testing was conducted with trained staff and/or health care workers (if reported). Univariate meta-analysis with random effects for sensitivity and specificity in cases where only a few studies were included (mixed population of symptomatic and asymptomatic children) did not show remarkable differences to the bivariate analysis. The results of all bivariate meta-analyses are summarised in Table 15.

• Download figure
• Open in new tab

Figure 6:

Summary receiver operating characteristic (SROC) plot of sensitivity and specificity of antigen tests for diagnosis of current SARS-CoV-2 infections in entire paediatric study populations irrespective of symptoms. Each circle represents the point estimate of an individual study, whereas the size of the circle correlates with the number of paediatric study participants (small circle: <100 participants, medium circle: between 100 and 500 participants, large circle: >500 participants). The pooled estimate (black dot) of the pair of sensitivity (Se) and specificity (Sp) is surrounded by its 95% confidence region (closed curve with short dashes) and prediction region (closed curve with long dashes). The estimation of the SROC curve is based on the bivariate approach by Rutter and Gatsonis [29].

• Download figure
• Open in new tab

Figure 7:

Summary receiver operating characteristic (SROC) plot of sensitivity and specificity of antigen tests for diagnosis of current SARS-CoV-2 infections in (a) symptomatic, (b) asymptomatic, and (c) mixed symptomatic and asymptomatic paediatric study populations. Each circle represents the point estimate of an individual study, whereas the size of the circle correlates with the number of paediatric study participants (small circle: <100 participants, medium circle: between 100 and 500 participants, large circle: >500 participants). The pooled estimate (black dot) of the pair of sensitivity (Se) and specificity (Sp) is surrounded by its 95% confidence region (closed curve with short dashes) and prediction region (closed curve with long dashes). The estimation of the SROC curve is based on the bivariate approach by Rutter and Gatsonis [29].

Limitations

The results presented in this review should be considered in light of the following limitations:

1. As visualized in the QUADAS-2 summary graph, the majority of studies were at unclear risk of bias in at least three of the four domains of the QUADAS-2 tool because of poor reporting by authors. Therefore, the results reported in this review should be interpreted with caution. The issue of insufficient reporting has been addressed previously and led to the publication of comprehensive guidance to authors evaluating the clinical performance of tests for SARS-CoV-2 [88].

2. We acknowledge that infectiousness as target condition is of higher practical relevance than current SARS-CoV-2 infection, which was chosen as the target condition of this review. While RT-PCR as the corresponding reference standard is a highly sensitive method that is used to detect the presence of viral ribonucleic acid (RNA) in a specimen, this does not necessarily indicate that infectious virus is present. Therefore, the actual transmission risk from individuals who tested RT-PCR positive remains unknown. Testing for infectiousness would allow to identify (and isolate) exclusively individuals who could pass the virus to others. However, while there have been attempts to use viral load (estimated from Ct values) or virus viability in cell culture as a proxy to determine infectious individuals, up to now, there is no adequate reference standard for infectiousness [82].

3. Screening testing programs of children in schools and kindergarten using antigen tests have been implemented in many countries. Specific aspects that define these use cases, such as sample collection in toddlers by laypersons or self-testing in schools performed by children, are likely to influence the real-life test performance but were not addressed in any of the studies included in our review. Furthermore, one should keep in mind that diagnostic accuracy is only one factor affecting the effectiveness of testing programs [89].

4. Only eight different antigen tests were included in our review, and four of these were only evaluated in one study each. Thus, the performance of most antigen tests under real-life conditions remains unknown. While the number of antigen tests is still small in the US due to more rigorous approval processes, the situation in Europe is different, since the current regulatory framework for market access in the Europe Union (EU) [90] allows self-declaration of regulatory conformity of SARS-CoV-2 antigen tests for professional use by manufacturers without the involvement of a third party. As of July 6, 2021, more than 500 antigen tests gained market access in the EU [91]. A recent laboratory-based study evaluated the sensitivity of 122 of these antigen tests using common SARS-CoV-2 specimens with varying viral concentrations [92]. Even under such ideal conditions, a wide range of sensitivities was observed, whereas 26 tests missed the study’s sensitivity criteria of 75% for specimens with high SARS-CoV-2 concentrations of around 10⁶ SARS-CoV-2 RNA/ml and higher corresponding to a Ct value of <25.

5. Diagnostic accuracy estimates reported in this review may not apply to future variants of SARS-CoV-2. Considering the reported recruitment periods, most individuals identified as RT-PCR positive were likely infected with the wild-type of SARS-CoV-2. The impact of new emerging variants of SARS-CoV-2 on the diagnostic accuracy of SARS-CoV-2 tests is unknown and requires thorough investigation.