Abstract
Background Navigating the rapidly growing body of scientific literature on the SARS-CoV-2 pandemic is challenging and ongoing critical appraisal of this output is essential. We aimed to collate and summarize all published systematic reviews on the coronavirus disease (COVID-19).
Methods Nine databases (Medline, EMBASE, Cochrane Library, CINAHL, Web of Sciences, PDQ-Evidence, WHO’s Global Research, LILACS and Epistemonikos) were searched from December 1, 2019 to March 24, 2020. Systematic reviews analyzing primary studies of COVID-19 were included. Two authors independently undertook screening, selection, extraction (data on clinical symptoms, prevalence, pharmacological and non-pharmacological interventions, diagnostic test assessment, laboratory and radiological findings) and quality assessment (AMSTAR 2). Meta-analysis on prevalence of clinical outcomes was performed.
Results Eighteen systematic reviews were included; one was empty. Using AMSTAR 2, confidence in the results of 13 reviews was rated as “critically low”, one as “low”, one as “moderate” and two as “high”. Symptoms of COVID-19 were (range values of point estimates): fever (82–95%), cough with or without sputum (58–72%), dyspnea (26–59%), myalgia or muscle fatigue (29–51%), sore throat (10–13%), headache (8– 12%) and gastrointestinal complaints (5–9%). Severe symptoms were more common in men. Elevated C-reactive protein (associated with lymphocytopenia) and lactate dehydrogenase, and slightly elevated aspartate and alanine aminotransferase, were commonly described. Thrombocytopenia and elevated levels of procalcitonin and cardiac troponin I were associated with severe disease. Chest imaging described a frequent pattern of uni- or bilateral multilobar ground-glass opacity. Only one review investigated the impact of medication (chloroquine) but found no verifiable clinical data. All-cause mortality ranged from 0.3% to 14%.
Conclusions Confidence in the results of most reviews was “critically low”. Future studies and systematic reviews should adhere to established methodologies. The majority of included systematic reviews were hampered by imprecise search strategy and no previous protocol submission.
Protocol registration This is an extension of a PROSPERO protocol (CRD42020170623); protocol available on Open Science Framework (https://osf.io/6xtyw).
1. Background
Spread of severe acute respiratory coronavirus 2 (SARS-CoV-2), the causal agent of COVID-19, has been characterized as a pandemic by the World Health Organization (WHO) and has triggered an international public health emergency [1]. The virus is easily transmitted through person-to-person contact and virus contamination of surfaces or objects [2,3]. As of April 7, 2020, more than 1,359,398 people have been confirmed to be infected with the virus and over 75,945 people have died as a result of severe complications of the disease [4]. These numbers are likely to be underestimates, as countries are unable to test all patients suspected of having the disease due to worldwide shortages of testing reagents [5]. Estimates put the economic impact of COVID-19 as surpassing trillions of dollars when global trade, travel and business costs are added to healthcare and public health expenses [6,7].
Given the novelty of the situation and dozens of daily publications in peer reviewed journals and preprints, there is an overwhelming avalanche of information from research articles to sift through. For healthcare professionals and government policy makers, the ability to sort through relevant research results, in order to clearly identify the most up- to-date and reliable information, is essential. Having filtered, accurate information at their disposal allows stakeholders to make timely decisions regarding the best course of action. Hence, our aim was to summarize the data from all available systematic reviews regarding COVID-19 in humans and to rigorously assess the level of confidence that we can have in this evidence.
2. Methodology
2.1. Protocol and Registration
This overview is a systematic review of systematic reviews. It is an extension of a recently submitted PROSPERO protocol (CRD42020170623) [8]. The protocol has also been published on Open Science Framework (OSF) (https://osf.io/6xtyw) for transparency. It was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [9]. The methodology used in this review was adapted from the Cochrane Handbook for Systematic Reviews of Interventions and also followed established methodological considerations for analyzing existing systematic reviews [10,11].
2.2. Ethics
Approval of a research ethics committee was not necessary as the study analyzed only publicly available articles.
2.3. Eligibility Criteria
Systematic reviews were included if they analyzed primary data from patients infected with SARS-CoV-2 as confirmed by RT-PCR or another pre-specified diagnostic technique. Eligible reviews covered all topics related to COVID-19, including, but not limited to those that reported clinical symptoms, diagnostic methods, therapeutic interventions, laboratory findings or radiological results. Both full manuscripts and abbreviated versions, such as letters, were eligible.
No restrictions were imposed on the design of the primary studies included within the systematic reviews, date of last search, whether or not the review included meta-analyses, or language. A systematic review was defined as a publication where: I. the search was conducted in at least two databases (at least one being electronic); and II. detailed description of the methods with explicit inclusion criteria was provided. We excluded narrative reviews as these are less likely to be replicable and are more prone to bias.
2.4. Information Sources
Nine databases were searched for eligible records published between December 1 2019 and March 24, 2020: Cochrane Database of Systematic Reviews via Cochrane Library, PubMed EMBASE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), Web of Sciences, LILACS (Latin American and Caribbean Health Sciences Literature), PDQ-Evidence, WHO’s Global Research on Coronavirus Disease (COVID-19), and Epistemonikos.
2.5. Search
The comprehensive search strategy for each database is provided in Appendix 1 and was designed and conducted in collaboration with an information specialist. All retrieved records were primarily processed in EndNote, where duplicates were removed, and records were then imported on Covidence Platform [12]. In addition to database searches, we screened reference lists of reviews included, after screening records retrieved via databases.
2.6. Study Selection
All searches, screening of titles and abstracts, as well as record selection, were performed independently by two investigators using the Covidence Platform [12]. Articles deemed potentially eligible were retrieved for full-text screening by two independent investigators. Discrepancies in both stages of screening were resolved by consensus. During screening, records published in languages other than English were translated by a native/fluent speaker.
2.7. Data Collection Process
We custom designed a data extraction table for the purpose of this study, which was piloted by two authors independently. Data extraction was performed independently by two authors. Conflicts were resolved by consensus or by consulting a third researcher.
2.8. Data Items
We extracted the following data: article identification data (authors’ name and journal of publication), range of period associated with the search, number of databases searched, population or settings considered, main results and outcomes observed, and number of participants. From Web of Science (Clarivate Analytics, Philadelphia, PA, USA) we extracted journal rank (quartile) and Journal Impact Factor (JIF).
We categorized the following as primary outcomes: all-cause mortality, need for and length of mechanical ventilation, length of hospitalization (in days), admission to intensive care unit (yes/no) and length of stay in intensive care unit.
The following were categorized as exploratory outcomes: diagnostic methods used for detection of the virus, male to female ratio, clinical symptoms, pharmacological and non-pharmacological interventions, laboratory findings (full blood count, liver enzymes, C-reactive protein, d-dimer, albumin, lipid profile, serum electrolytes, blood vitamin levels, glucose levels, and any other important biomarkers), and radiological findings (using radiography, computed tomography, magnetic resonance imaging or ultrasound).
We also collected data on reporting guidelines and requirements for publication of systematic reviews and meta-analyses from journal websites where included reviews were published
2.9. Quality Assessment in Individual Reviews
Two researchers independently assessed the quality of the reviews using the “A MeaSurement Tool to Assess systematic Reviews 2 (AMSTAR 2)”, designed to assess systematic reviews that include both randomized and non-randomized studies. Adherence to each item was rated as follows: yes, no, partial yes, not applicable or unclear. We rated overall confidence in the results of the review as “critically low”, “low”, “moderate” or “high”, according to the AMSTAR 2 guidance based on seven critical domains defined by AMSTAR 2 authors [13]. If a review did not find any studies to include (i.e., it was an “empty review”), we scored each individual item, but we did not conduct summary confidence assessment for such reviews.
One of the included systematic reviews was conducted by some members of this author team. To prevent risk of bias in assessing methodological quality of this particular review, it was assessed independently by two authors who did not co-author it.
2.10. Synthesis of Results
For data synthesis, we prepared a table summarizing each systematic review. Graphics illustrating mortality rate and clinical symptoms were created. We then prepared a narrative summary of the methods, findings, study strengths and limitations.
For analysis of prevalence of clinical outcomes, we extracted data on the number of events and total number of patients to perform proportional meta-analysis using RStudio© software, with the “meta” package (version 4.9-6), using the “metaprop” function for reviews that did not perform meta-analysis, excluding case studies because of the absence of variance. For reviews that did not perform meta-analysis, we presented pooled results of proportions with their respective confidence intervals (95%) by the inverse variance method with a random-effects model, using the DerSimonian-Laird estimator for τ2. We adjusted data using Freeman-Tukey double arcosen transformation. Confidence intervals were calculated using the Clopper-Pearson method for individual studies. Forest plots were generated showing data from individual systematic reviews, but results were not pooled because some studies were included in multiple reviews. We developed the using RStudio© software, with the “metafor” package (version 2.1-0) and “forest” function.
3. Results
Our search retrieved 1,063 publications, of which 175 were duplicates. Most publications were discarded after title and abstract analysis (n = 860), 28 were selected for full text screening and 18 were included in the final analysis (Figure 1) [14-31]. Reference list screening did not retrieve any additional systematic reviews.
3.1. Characteristics of included reviews
Summary features of the 18 systematic reviews are presented in Table S1. Included reviews were published in 14 different journals. Only four of these journals had specific requirements for submission of systematic reviews (with or without meta-analysis): European Journal of Internal Medicine, Journal of Clinical Medicine, Ultrasound in Obstetrics and Gynecology and Clinical Research in Cardiology. Two journals reported that they published only invited reviews (Journal of Medical Virology and Clinica Chimica Acta). Three reviews were published as a letter, and one was labelled as a rapid review (Table S2).
All the reviews were published in English in a first quartile (Q1) journal, with JIF (Journal Impact Factor) ranging from 1.692 to 6.062. The 18 systematic reviews covered more than 120 individual primary studies, involving in total at least 60,000 participants. Ten systematic reviews included meta-analyses. Primary studies included in the reviews were published between December 2019 and March 18, 2020, and comprised case reports, case series, cohorts and other observational studies. We did not find any review with randomized clinical trials. In these reviews, systematic literature searches were performed from 2019 (entire year) up to March 9, 2020.
3.2. Population and study designs
Most of the reviews analyzed data from patients with COVID-19 who developed pneumonia, acute respiratory distress syndrome (ARDS) or any other correlated complication. Two reviews were based on in vitro investigations, one review aimed to evaluate the effectiveness of using surgical masks on preventing transmission of the virus [25], one review was focused on pediatric patients [23] and one review investigated COVID-19 in pregnant women (Figure 1) [26]. Most reviews assessed clinical symptoms, laboratory findings or radiological results.
3.3. Systematic review findings
The summary of findings from individual reviews is shown in Table S2. Overall, all-cause mortality ranged from 0.3% to 14% (Figure 2).
3.3.1. Clinical symptoms
Seven reviews described the main clinical manifestations of COVID-19 [14,15,22,23,24,28,30]. Three of them provided only a narrative discussion of symptoms [14,23,24]. Four reviews performed a statistical analysis of the incidence of different clinical symptoms. Overall, symptoms in patients with COVID-19 were (range values of the point estimates): fever (82–95%), cough with or without sputum (58– 72%), dyspnea (26– 59%), myalgia or muscle fatigue (29–51%), sore throat (10–13%), headache (8–12%), gastrointestinal disorders, such as diarrhea, nausea or vomiting (5.0–9.0%), and others (including, in one study only: dizziness 12.1%) (Figure 3-9). Three reviews assessed cough with and without sputum together; only one review assessed sputum production itself (28.5%).
3.3.2. Diagnostic aspects
Three reviews described methodologies, protocols and tools used for establishing the diagnosis of COVID-19 [14,23,27]. Use of respiratory swabs (nasal or pharyngeal) or blood specimens to assess the presence of SARS-CoV-2 nucleic acid using RT-PCR assays was the most commonly used diagnostic methods mentioned in the included studies. These diagnostic tests are widely used, but their precise sensitivity and specificity remain unknown. One review included a Chinese study in which there was clinical diagnosis with no confirmation of SARS-CoV-2 infection (patients were diagnosed with COVID-19 if they presented with at least 2 symptoms suggestive of COVID-19, as well as laboratory and chest radiography abnormalities) [23].
3.3.3. Therapeutic possibilities
Pharmacological and non-pharmacological interventions (supportive therapies) used in the treatment of patients with COVID-19 were reported in five reviews [16,17,23,24,27]. Most reviews were descriptive, as the original studies did not assess the impact of therapies. Antivirals have been used empirically for COVID-19 treatment with the most commonly used being protease inhibitors (lopinavir, ritonavir, darunavir), nucleoside reverse transcriptase inhibitor (tenofovir), nucleotide analog (remdesivir, galidesivir, ganciclovir), and neuraminidase inhibitors (oseltamivir). Umifenovir, a membrane fusion inhibitor, was investigated in several studies [16,24]. Possible supportive interventions analyzed were different types of oxygen supplementation and breathing support (invasive or non-invasive ventilation). Use of antibiotics, both empirically and to treat secondary pneumonia, was reported in some studies. One review specifically assessed evidence on the efficacy and safety of the anti-malaria drug chloroquine [17]. It identified 23 ongoing trials investigating the potential of chloroquine as a therapeutic option for COVID-19, but no verifiable clinical outcomes data. Use of mesenchymal stem cells, antifungals and glucocorticoids were seldom described.
3.3.4. Laboratory and radiological findings
Of the 18 reviews included in this overview, eight analyzed laboratory parameters in patients with COVID-19 [16,19-24,28]. Elevated C-reactive protein levels, associated with lymphocytopenia, elevated lactate dehydrogenase, as well as slightly elevated aspartate and alanine aminotransferase (AST, ALT) were commonly described. Lippi et al. assessed the variation of cardiac troponin I (cTnI), procalcitonin, and platelet count between severe COVID-19 patients and non-severe COVID-19 patients [19-21]. Elevated levels of procalcitonin and cTnI were more likely to be associated with a severe disease course (requiring intensive care unit admission and intubation). Furthermore, thrombocytopenia was frequently observed in patients with complicated COVID-19 infections.
Chest imaging (chest radiography and/or computed tomography) features were assessed in six reviews, all of which described a frequent pattern of local or bilateral multilobar ground-glass opacity [16,23,24,28-30]. Septal thickening, bronchiectasis, pleural and cardiac effusions, halo signs and pneumothorax were observed in patients suffering from COVID-19.
3.4 Quality of evidence in individual systematic reviews
Table S3 shows the detailed results of the quality assessment of 18 systematic reviews, including assessment of individual items and summary assessment. Overall confidence in results of reviews was assessed for 17 reviews; for one empty review summary assessment was not made [25]. Using AMSTAR 2 criteria, confidence in the results of 13 out of 17 reviews with included studies was rated as “critically low”, one as “low”, one as “moderate” and two as “high”. Common methodological drawbacks were: omission of prospective protocol submission or publication; use of inappropriate search strategy: lack of independent and dual literature screening and data-extraction (or methodology unclear); absence of explanation for heterogeneity among the studies included; lack of reasons for study exclusion (or rationale unclear).
Risk of bias assessment, based on a clearly reported methodological tool, and quality of evidence appraisal, in line with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) method, were reported only in one review.16 Systematic reviews should perform a risk of bias assessment of included primary studies, but only five (27.7%) of the included reviews presented a table summarizing bias. Review authors used a varied number of risk of bias tools. Of these, four reviews classified studies as being of “fair” or “appropriate quality” and one review classified studies as having a high risk of bias. One review mentioned the risk of bias assessment in the methodology but did not provide any related analysis. It is worth mentioning that the reviews used a varied amount of risk of bias tools.
4. Discussion
This overview of systematic reviews, to our knowledge the first of its kind on COVID-19, included 18 reviews published up to March 24, 2020, and involving more than 60,000 patients. Ten reviews included meta-analyses. The reviews presented data on clinical manifestations, laboratory and radiological findings and interventions. We found no systematic reviews on the utility of diagnostic tests.
Symptoms were reported in seven reviews; the majority of the patents had fever, cough, dyspnea, myalgia or muscle fatigue and gastrointestinal disorders such as diarrhea, nausea or vomiting. Olfactory dysfunction (anosmia or dysosmia) has been reported in the scientific literature to occur in some patients infected with COVID-19 but this was not reported in any of the assessed systematic reviews [32,33,34]. During the SARS outbreak in 2002, there were reports of impairment of sense of smell associated with the disease [35,36]. Nasal obstruction and post-viral olfactory disorders are possible reasons for this symptom [37].
The reported mortality rates ranged from 0.3 to 14% in the included reviews. Mortality estimates are influenced by the transmissibility rate (basic reproduction number), availability of diagnostic tools, notification policies, asymptomatic presentations of the disease, resources for disease prevention and control and treatment facilities. Even with more than 1,359,398 people infected as of April 7, 2020 [4], variability in the mortality rate fits the pattern of emerging infectious diseases [38]. Furthermore, the reported cases do not consider asymptomatic cases, mild cases where individuals have not sought medical treatment, and the fact that many countries have limited access to diagnostic tests or have implemented testing policies only very recently [39]. The lack of reviews assessing diagnostic testing (sensitivity, specificity and predictive values of RT-PCT or immunoglobulin tests), and the preponderance of studies that assessed only symptomatic individuals, means that considerable imprecision remains around the calculated mortality rate.
Few reviews included treatment data. Those that did described studies offering a very low level of evidence: usually small, retrospective studies with very heterogenous populations. The efficacy of pharmacological interventions therefore remains extremely unclear. Encouragingly, on April 7, 2020 there were 366 clinical trials for COVID-19 registered on ClinicalTrials.gov, with many pharmacological interventions being tested, so hopefully this gap in the evidence will be filled soon [40].
Seven reviews analyzed laboratory parameters. Elevated CRP, lactate dehydrogenase, AST and ALT were commonly reported; elevated cTnI and procalcitonin, and lower platelet count, appear to be associated with more severe disease. This information may be useful for clinicians who attend patients suspected of COVID-19 in emergency services worldwide, for example, to assess which patients need to be reassessed more frequently. Further research is required to assess their relation to mortality and other patient outcomes, as well as prognosis, in order to investigate whether they can contribute to decision-making.
Most systematic reviews scored poorly on the AMSTAR 2 critical appraisal tool for systematic reviews. Thirteen of the 17 included reviews achieved the lowest rating of “critically low” in the confidence readers can have in the results of the review. In contrast, only two reviews achieved the top rating of “high” confidence in the results. Most of the original studies included in the reviews were case series and case reports, which impacts on the quality of evidence. This has major implications for clinical practice and use of these reviews in evidence-based practice and policy. Clinicians, patients and policymakers can only have the highest confidence in systematic review findings if high-quality systematic review methodologies are employed. The urgent need for information during a pandemic does not justify poor quality reporting.
A limitation of our study is that for AMSTAR 2 assessment we relied on information available in publications; we did not attempt to contact study authors for clarifications or additional data. Appraisal of methodological quality of three reviews was hindered because they were published as letters, labelled as rapid communications. As a result, various details about their review process were not included, leading to AMSTAR 2 questions being answered as not reported and thus to low confidence scores. Full manuscripts might have provided additional information that could have led to higher confidence in the results. In other words, low scores could reflect incomplete reporting, not necessarily low quality review methods. In seeking to make their review available more rapidly and more concisely, methodological details may have been omitted. This reflects a general issue during a crisis where speed and completeness must be balanced. Given that some included reviews scored “high”, the two factors are not mutually exclusive. However, this requires proper resourcing and commitment to ensure users of systematic reviews can have high confidence in the results.
Despite this limitation, this overview adds to the current knowledge by providing a comprehensive summary of all the currently available evidence about COVID-19 from published systematic reviews. This overview followed strict methodological criteria, including a comprehensive and sensitive search strategy and a standard tool for methodological appraisal of systematic reviews. Another potential limitation is that several authors from this study have contributed to one of the reviews identified. To reduce risk of any bias, two authors who did not contribute to the review in question undertook the assessment of its quality and limitations.
Further research is of the utmost importance to assess the impact of antivirals, chloroquine and other interventions on patient outcomes. Additionally, it is essential to determine the accuracy of available diagnostic tests. Understanding the role of different biomarkers to predict clinical outcomes, and whether they could be used in association with validated scores, such as Sequential Organ Failure Assessment (SOFA) score, to help the decision of choosing patients for scarce intensive care unit beds would be very useful.
5. Conclusion
In conclusion, by conducting this overview of systematic reviews, we have quantified the overall incidence of clinical manifestations of patients infected with SARS-CoV-2 and described its more common laboratory and radiological abnormalities. The evidence is sufficient to guide medical doctors and health-related professionals on the construction of clinical rationale of non-tested cases, but insufficient to provide any specific endorsement of pharmacological and non-pharmacological interventions. In order to fill evidence-based gaps, high quality primary studies should be carried out initially, so that well-designed systematic reviews can be performed in the future.
Data Availability
All data involved in this submission is fully available.
Declarations
Ethics approval and consent to participate
Not required as data was based on primary studies.
Consent for publication
Not applicable
Availability of data and materials
All data collected and analyzed within this study are available from the corresponding author on reasonable request.
Competing interests
The authors declare no conflict of interest.
Funding
This research received no external funding
Authors’ contributions
IJBN conceived the research idea and worked as a project coordinator. MSM, DPOM, TCVG, HMA, IW, AM, LP, VTC, IZG, TPP and ANA were involved in data curation, formal analysis, investigation, methodology, and initial draft writing. IJBN, MSM, DPOM, SF, NLB and LP analyzed, processed, interpreted the data and wrote the final manuscript draft and also revised it afterwards. All affiliated authors provided feedback to the manuscript, which was coordinated by IJBN and TCVG.
Acknowledgements
We thank Catherine Henderson DPhil from Swanscoe Communications for pro bono medical writing and editing support. We acknowledge support from Covidence Team, specifically Anneliese Arno. We thank the whole International Network of Coronavirus Disease 2019 (InterNetCOVID-19) for their commitment and involvement. Members of the InterNetCOVID-19 are listed in Appendix 2. We thank Pavel Cerny and Roger Crosthwaite for guiding the team supervisor (IJBN) on human resources management.
This file with figures has been provided by the authors to give readers additional information about their work
This file with tables has been provided by the authors to give readers additional information about their work
Footnotes
e-mail: israeljbn{at}ufmg.br; milenamarc{at}gmail.com
omathuna.6{at}osu.edu
thilo.vongroote{at}gmail.com
dr.hebatullah.mohamed{at}gmail.com
Ishanka.Weerasekara{at}uon.edu.au
ana.marusic{at}mefst.hr, tinapoklepovic{at}gmail.com, izakarijagrkovic{at}yahoo.com
livia.puljak{at}unicath.hr
vinicius_civile{at}yahoo.com.br; alvaro.atallah{at}gmail.com
santino_filoso{at}hotmail.com
robertobragazzi{at}gmail.com