Efficacy of Chloroquine or Hydroxychloroquine in COVID-19 Patients: A Systematic Review and Meta-Analysis

Inclusion and Exclusion Criteria

We followed Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA)[20] guidelines for reporting systematic review and meta-analysis of observational studies guidelines. We included 1) RCTs or 2) cohorts or case control studies reporting on adjusted effect estimates of the association between Hydroxychloroquine and the following endpoints: (1) short term mortality, (2) mechanical ventilation/ ICU admission and (3) viral clearance.

Literature Search

The literature was searched by a medical librarian for the concepts of CQ or HCQ combined with COVID-19. The search strategies were created using a combination of keywords and standardized index terms. Searches were run through July 7, 2020 in Ovid EBM Reviews, Ovid Embase (1974+), Ovid Medline (1946+ including epub ahead of print, in-process & other non-indexed citations), Scopus (1970+) and Web of Science (1975+). Search strategies are provided in the supplement table 1. We also searched for unpublished manuscripts using the medRxiv services operated by Cold Spring Harbor Laboratory and Research Square preprints. In addition, we searched Google Scholar and the references of eligible studies and review articles.

Two reviewers independently identified eligible studies (ZK and OA) and 4 reviewers (ZK, MG, OA, HT) extracted the data into a pre-specified data collection form. A senior reviewer verified all data included in the analyses (IT).

Two reviewers (ZK and OA) independently assessed risk of bias for each study using the (RoB 2) of the Cochrane risk-of-bias tool for randomized trials [21] and the Newcastle–Ottawa instrument for cohort studies and case–control studies [22]. Reviewers judged each criterion for risk of bias and resolved any disagreements by discussion with a third senior reviewer. We assessed the certainty of evidence for each of our outcomes using the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach [23, 24]. This method evaluates the certainty of evidence by assessing the following domains: Limitations, indirectness, inconsistency, imprecision, and publication bias.

Statistical analysis

We pooled studies using the DerSimonian-Laird random-effects model (and constructed corresponding forest plots). Pooled adjusted effect estimates (ORs and HRs) were obtained by combining the estimates of log (adjusted effect estimate) from each study. Endpoints that we considered a priori for the meta-analysis were: (1) short term mortality, (2) mechanical ventilation/ ICU admission and (3) viral clearance. We evaluated heterogeneity using the I² statistic which estimates the variability percentage in effect estimates that is due to heterogeneity rather than to chance—the larger the I², the greater the heterogeneity. We conducted sensitivity analyses to assess the impact of (1) risk of bias in included studies, and (2) the selection of study population (general populations vs. specific populations), on the overall estimate of effect. We constructed funnel plots and performed an Egger precision-weighted linear regression test as a statistical test of funnel plot asymmetry and publication bias. All analyses were conducted using Stata version 16 statistical software (StataCorp, College Station, Texas).

Included studies

Out of 1,896 papers screened for eligibility, 5 RCTs [25-29], 14 cohort studies [30-43] were included (Figure 1) with a total of 20,263 patients. Characteristics of included studies are described in Tables 1 and 2. Characteristics of the patients in each study can be found in Supplementary Table 5. All studies reported on patients hospitalized with COVID-19. Alberici et al. also included out-patients (39%). Most of the studies included patients presenting with varying levels of severity; Ip et al. [33], Magagnoli et al. [35], Rosenberg et al. [39] included patients who were hypoxic, t or were admitted to the ICU, in addition to patients with relatively more stable vitals. Paccoud et al. [38] included patients with varying pneumonia severity. Yu et al. only studied patients with critical disease whereas Geleris et al. [32] and Mahevas et al. [36] excluded them.

• Download figure
• Open in new tab

Figure 1.

PRISMA Flow Diagram of Eligible Studies

• Download figure
• Open in new tab

Figure 2.

Association between Hydroxychloroquine and short-term mortality in COVID-19 patients: (All cohorts and 1 RCT)

• Download figure
• Open in new tab

Figure 3.

Association between Hydroxychloroquine and short-term mortality in COVID-19 patients: (Excluding studies at high risk for bias)

• Download figure
• Open in new tab

Figure 4.

Association between Hydroxychloroquine and AZ combination and short-term mortality in COVID-19 patients

The quality of the observational studies was assessed using the Newcastle-Ottawa Scale (Supplementary Table 11). With regards to patient selection, all but four studies had adequate representation of a general population with COVID-19 infection. Alvarez et al. [31] and Alberici et al.[30] studied hemodialysis patients, Yu et al. [42] studied patients with severe COVID-19 infection, and Kuderer et al. [34] studied patients with cancer. With regards to comparability, three studies [30, 31, 37] did not adequately adjust for confounders in their analyses. Given the relatively short course of disease, all studies were considered to have had a satisfactory follow-up duration. Three studies [30, 31, 42] that assessed mortality were considered at high risk of bias, and a sensitivity analysis was performed by excluding these studies to assess heterogeneity. The results of the Cochrane ROB assessment for RCTs are found in supplementary table 12. Of the 5 included RCTs 3 were considered high risk [27-29], one RCT was low risk [25], and one had no available manuscript for quality assessment [26].

Some studies were initially included in our review as eligible studies but were later excluded for various reasons. Chen Z et al. [44] reported time to clinical recovery and changes in radiologic parameters. Feng et al. [45] reported disease progression in patients receiving chloroquine, whereas only Esper [46] reported hospitalization in outpatients taking HCQ compared to controls (1.9 vs 5.4 P<0.0001). Four studies were excluded over serious methodology concerns [47-50] and one study was retracted due to concerns over the validity of the patient information [51]. GRADE was used to assess the certainty of evidence for each outcome. The findings were then summarized, along with the overall certainty of evidence (Table 3).

Treatment efficacy

Main findings

Our systematic review and meta-analysis included 12 cohorts and 1 RCT, which addressed the association between HCQ therapy in improving and mortality in patients hospitalized with COVID-19 disease. Among a total of 19573 patients, we found with moderate level of certainty, that HCQ monotherapy did not reduce short-term mortality among COVID-19 patients, which remained statistically non-significant even after excluding three studies at high risk of bias. These observations did not change when we included in the model only the cohort studies. Moreover, we found that the use of combination of HCQ and AZ was associated with a trend of increased mortality. The study by Kruderer et al. analyzed cancer patients and demonstrated higher mortality than other studies creating significant heterogeneity. Excluding this study did not decrease the risk of mortality. Because of the limited number of studies and/or high risk of bias, we could not meta-analyze other clinical outcomes such as viral clearance, risk of ICU admission, and need for mechanical ventilation.

Our findings of lack of efficacy of HCQ in the inpatient clinical setting despite its effective in vitro inhibitory actions against SARS-CoV-2 virus is consistent with previous observations with other viral illnesses. Numerous studies demonstrated significant in vitro inhibitory effects of CQ/HCQ against coronaviruses and non-coronaviruses [11]. For example, CQ at half-maximum effective concentration (EC₅₀) of 8.8 ± 1.2 µM effectively inhibited SARS-CoV replication in Vero E6 cells [3]. CQ was also shown to inhibit MERS-CoV and alphacoronavirus HCoV-229E replication in vitro in a dose dependent manner [4]. Likewise, CQ/HCQ exhibited in vitro inhibitory effects against several other viruses such as HIV-1, influenza, dengue, Ebola, Zika, chikungunya viruses and others [2, 5-10]. Although CQ/HCQ have shown consistent broad spectrum in vitro antiviral effects, their in-vivo and clinical antiviral effects were disappointing. For instance, CQ was ineffective in preventing or ameliorating influenza following viral challenge in mouse and ferret models [5] and did not prevent influenza infection in a randomized, double-blind placebo-controlled trial in humans [52]. CQ also resulted in worse outcomes in a guinea pig model of Ebola infection [7] and was shown to enhance chikungunya viral infections in different animal models including non-human primates [53]. Moreover, CQ was s ineffective in improving the course of chikungunya in humans [53, 54]. CQ was also tested in a randomized controlled trial of 307 patients with dengue virus and failed to reduce duration of viremia or NS1 antigenemia [55]. In the case of HIV-1, the use of CQ/HCQ was inconclusive and hence they were not endorsed for routine use in the treatment of HIV-1 infection [56].

Possible reasons for lack of efficacy of HCQ in the treatment of COVID-19 disease

The discrepancy between the observed in vitro anti-SARS-CoV-2 effects of CQ/HCQ and the lack of efficacy in clinical studies, which mirrors previous observations with other viral infections could be due to three main reasons:

First, most of in vitro studies employ pre-treatment protocols where cells are treated with the drugs before infecting them with the tested virus. In vitro studies that compared pre-treatment and post-infection treatment have shown that CQ/HCQ have less effective antiviral activities if added after infection [4, 16, 57]. For example, Vincent et al. [57] showed that e CQ at 0.1, 1.0 and 10 µM added 20-24 hours before the infection with SARS-CoV decreased infectivity by 28%, 53% and 100%. However, if CQ is added 3-5 hours after infecting the cells, higher concentrations of CQ of up to 50 µM were needed to decrease infectivity [57]. This may raise the possibility that chronic or prophylactic use of CQ/HCQ may reduce the risk of acquiring SARS-CoV-2 infection. However, a recent large population study of 14,250 individuals showed that chronic use of HCQ was not protective against SARS-CoV-2 infection [58].

Second, a wide range of EC₅₀ values were reported for CQ and HCQ and in the case of SARS-CoV-2, the EC₅₀ for CQ ranges between 1.13 and 7.36 µM and between 0.72 and 17.31 µM for HCQ [11]. It is worth noting that the lowest EC₅₀ of 0.72 µM for HCQ was reported by Yao et al in their post-infection experiments, was different from the lowest EC₅₀ of 5.85 µM in their pre-treatment experiments [16]. None of the other investigators reported such lower EC₅₀ for CQ of HCQ with SARS-CoV-2 or any other viruses. Achieving adequate blood and tissue drug concertation is essential for proper antiviral activity In mice, a high dose of 90 mg/kg twice a day of CQ was necessary to achieve a steady state blood levels of 2.5 µg/ml [8]. Considering high doses of CO/HCQ in COVID-19 patients can be associated with increased adverse events as shown in a recent RCT where high dose CQ has been shown to be associated with significant toxicity in COVID-19 patients [59]. In addition, it is not clear what is the optimal CQ or HCQ blood levels for effective antiviral action since the suggested levels were based on widely differing in vitro EC₅₀ estimates. In a study of 40 HIV-1 patients treated with HCQ 800 mg/day for 8 weeks, the range of HCQ blood concentration was 0.27-1.0 µg/ml [60]. Only those HIV-1 patients who achieved the highest HCQ blood concentrations had favorable response to HCQ [60]. There are only two small studies that looked at the pharmacokinetics of HCQ in COVID-19 patients [18, 60, 61]. In the first study Gautret et al. determined the mean HCQ levels in 20 patients treated with HCQ 600 mg/day and was 0.46 µg/ml [18]. This blood concentration is lower than the lowest estimated levels of 0.48 µg/ml based on the lowest effective in vitro concentration of 0.72 µM. Perinel et al showed that only 61% of 13 patients treated with HCQ 600 mg/day achieved what they considered the minimum therapeutic concentration of 1 µg/ml with the mean time to reach this concentration of 2.7 days [61]. Based on published data and their own, Balevic et al. found that the average serum/plasma HCQ concentration were below the lowest antiviral target levels for SARS-CoV-2 of 0.48 µg/ml in all studies [62]. These studies indicate that current HCQ dosing is probably suboptimal to achieve adequate blood levels necessary for effective antiviral activity.

Third, Antimalarials exhibit anti-inflammatory and immunomodulatory effects where they can decrease the production of pro-inflammatory cytokines, and improve endothelial function and reduce prothrombotic state [12, 13]. These effects would be very beneficial in patients with severe COVID-19 disease; however, HCQ reduce the affinity of toll-like receptor 7 and 9 (TLR7 and TLR9) to viral RNA and also inhibit cyclic GMP-AMP synthase (cGAS) pathway and hence it inhibits type I interferon response which is the first defense line of the innate immune system against viral infections [12]. This effect might counteract the direct antiviral effects of HCQ and reduce its efficacy in treating COVID-19 disease.

Further research is needed to address these important issues to improve the clinical utility of CQ/HCQ for the treatment of COVID-19 disease, which should include exploring alternate administration routes like intra nasal application and inhalation therapy.

Safety of CQ/HCQ in the context of COVID-19 disease

Our meta-analysis not only revealed lack of efficacy of HCQ in improving the outcomes of COVID-19 patients but also suggests possible increased risk of mortality when used in combination with AZ. Several studies have shown increased risk of cardiac toxicity among COVID-19 patients treated with CQ/HCQ. Our group have recently conducted a meta-analysis on CQ/HCQ induced cardiac toxicity in COVID-19 patients, which revealed [63] increased risk of QTc prolongation and discontinuation of drug due to QT prolongation. In addition, CQ/HCQ was associated with a clinically significant risk malignant arrhythmias and cardiac arrest [59]. It has also been a common practice to use HCQ in combination with AZ for COVID-19 during the current pandemic. AZ has s been linked to increased risk of sudden cardiac death [63, 64]. Hence, the concomitant use of CQ/HCQ and AZ or other QT prolonging agents could potentially increase the risk of serious cardiac arrhythmias and death. Increased risk of 30-day cardiac death, angina and heart failure complications associated with the combination therapy of HCQ and AZ has also been reported in a recent preprint of a large population study of 323,122 patients [64]. Our findings are consistent with the IDSA recommendations on the use of CQ/HCQ in COVID-19 disease [65], and the recent systematic reviews [62-64]. However, our study has several advantages over the previous reviews (supplementary table 13). First, it is the only study that included both qualitative and quantitative analyses. Second, it has the largest number of identified studies and therefore patient population because our search is the most up to date. Third, in contrast to all previous studies, we included only studies that reported adjusted effect estimates and therefore we avoided including studies at high risk of bias due to confounding. Fourth, we provided adequate assessment of the certainty of evidence using the GRADE classification. Finally, we offered a comprehensive discussion regarding the probable mechanisms of lack of efficacy of HCQ in COVID-19, which will inform and stimulate further research in this area.