Quantifying treatment effects of hydroxychloroquine and azithromycin for COVID-19: a secondary analysis of an open label non-randomized clinical trial ====================================================================================================================================================== * Andrew A. Lover ## Abstract Human infections with a novel coronavirus (SARS-CoV-2) were first identified via syndromic surveillance in December of 2019 in Wuhan China. Since identification, infections (coronavirus disease-2019; COVID-19) caused by this novel pathogen have spread globally, with more than 250,000 confirmed cases as of March 21, 2020. An open-label clinical trial has just concluded, suggesting improved resolution of viremia with use of two existing therapies: hydroxychloroquine (HCQ) as monotherapy, and in combination with azithromycin (HCQ-AZ). [3, 4]. The results of this important trial have major implications for global policy in the rapid scale-up and response to this pandemic. The authors present results with p-values for differences in proportions between the study arms, but their analysis is not able to provide effect size estimates. To address this gap, more modern analytical methods including survival models, have been applied to these data, and show modest to no impact of HCQ treatment, with more significant effects from the HCQ-AZ combination, potentially suggesting a role for co-infections in COVID-19 pathogenesis. The trial of Gautret and colleagues, with consideration of the effect sizes, and p-values from multiple models, does not provide sufficient evidence to support wide-scale rollout of HCQ monotherapy for the treatment of COVID-19; larger randomized studies should be considered. These data also suggest further randomized-controlled studies of HCQ-AZ combination therapy should be undertaken. *K*eywords * COVID-19 * Emerging pathogens * Pharmaceutical therapies * Clinical trials * Secondary analyses ## 1 Introduction Evidence-based public health programming is essential for global pandemic planning, and optimization of resources. However, unadjusted analyses may provide distorted estimates, or not full utilize scarce clinical data, especially in with consideration of intention-to-treat analyses, where all persons enrolled are analysed [5, 6]. Finally, effect size estimates allow for the magnitude of potential clinical impact to considered in policy adoption, as statistical significance may not be biologically important [7]. ## 2 Data and Analysis All analyses was performed using Stata 16.1 (College Station, TX, USA). Standard 95% confidence interval were used; model parsimony was assessed using Akaike and Bayesian information criteria (AIC/BIC). All models were adjusted for age and sex. Assessing the predictive power of logistic models used Tjur’s ![Graphic][1]. [8]. ### 2.1 Data source Data were obtained from [4], and the Supplemental Table 1 was digitized using Tabula software, with subsequent hand-validation. Data for the six patients who were lost-to-follow-up (LTF) were manually entered into a database. View this table: [Table 1:](http://medrxiv.org/content/early/2020/04/10/2020.03.22.20040949/T1) Table 1: Risk ratios for clearance of virema, by day six, using binary regression (Primary outcome). (N=30). View this table: [Table 2:](http://medrxiv.org/content/early/2020/04/10/2020.03.22.20040949/T2) Table 2: Odds ratios for clearance of viremia, by day six, from a Firth penalized likelihood regression (Primary outcome). (N=30). View this table: [Table 3:](http://medrxiv.org/content/early/2020/04/10/2020.03.22.20040949/T3) Table 3: Hazard ratios for time-to-first negative PCR, using flexible parametric models. (N=36). (Secondary Outcome) ### 2.2 Primary outcome The primary outcome as reported by the authors “The primary endpoint was virological clearance at day-6 post-inclusion.” An optimal analysis for this endpoints in a binary regression which avoids many of the potential biases in logistic models when outcomes are common [9]. ### 2.3 Secondary outcome The stated “Secondary endpoint was virological clearance overtime during the study period.” and standard Cox survival time models and Royston-Parmar flexible parameteric models [10] were used to capture the time-to-first negative PCR (with a Ct threshold of >=35). The incorporation of the censored patients was as per standard methods. ## 3 Results The sequence of confirmed viremia via PCR is shown in (Fig. 1), and the LFT patients are at the bottom of the figure. The primary outcome was assessed using binary regressions to provide relative risks for clearance of viremia between the study arms. The main effect of interest (that of all combined HCQ-treated patients versus control), shows a marginally significant risk ratio of 3.84 (95 % CI 1.02 - 14.42, p= 0.047). Analysis of the separate HCQ and HCQ+AZ outcome was not possible due to quasi-separation of the model. ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/04/10/2020.03.22.20040949/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2020/04/10/2020.03.22.20040949/F1) Figure 1: Sequence plot of enrolled patients. (N= 42). ![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/04/10/2020.03.22.20040949/F2.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2020/04/10/2020.03.22.20040949/F2) Figure 2: Unadjusted Kaplan-Meier plot of all enrolled patients (N= 42). To address the limitations of these models, Firth penalized-likelihood model were used, which deal well with separation and quasi-separation. [11] To assess the secondary outcome, Kaplan-Meier and Cox models were use initially to compare the time-to-event with adjustment for covariates. Due severe proportional hazard violations in the HCQ-AZ group, flexible parametric models were used to estimate effects. ## 4 Discussion and Conclusions Together these results, especially in consideration of the loss to followup of six patients, do not provide sufficient evidence to support HCQ monotherapy for the treatment of COVID-19. This analysis is not without limitations: interpretation of the presented dataset may be incorrect; the analysis models for the primary and secondary outcome endpoints is inferred from the original authors’ description and power calcualtions (case-control design); and not all covariates were available in the original data [12]. However, taken together, this analysis does suggest further studies of HCQ-AZ combination therapy should be prioritized with great haste. The rapid increase in confirmed infections within the last few days suggests that the pandemic is accelerating, and there are major opportunity costs associated with all choices [13]; and rapid science will be critical for progress [14]. ## Data Availability Data sources, statistical code, and supporting documents are available here: [https://github.com/andrewlover/HCQ\_AZ\_COVID\_19](https://github.com/andrewlover/HCQ_AZ_COVID_19) ## Revision History View this table: [Table4](http://medrxiv.org/content/early/2020/04/10/2020.03.22.20040949/T4) ## Footnotes * * [https://www.umass.edu/sphhs/person/faculty/andrew-lover](https://www.umass.edu/sphhs/person/faculty/andrew-lover) * The author stands by all analytical and statistical aspects of this preprint. However, subsequent to this analysis, further details of the original study have been released-with major uncertainties in study design, reporting, choice of endpoints, and most importantly, data integrity [1, 2]. Therefore, all results from the original study should be viewed with considerable skepticism. * Received March 22, 2020. * Revision received April 1, 2020. * Accepted April 10, 2020. * © 2020, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/) ## References 1. [1].Elisabeth Bik. Thoughts on the Gautret et al. paper about Hydroxychloroquine and Azithromycin treatment of COVID-19 infections. Technical report, March 2020. Library Catalog: scienceintegritydigest.com. 2. [2].Pubpeer. Pubpeer: Hydroxychloroquine and Azithromycin as a treatment of COVID-19: preliminary results of an open-label non-randomized clinical trial, March 2020. 3. [3].Hydroxychloroquine and azithromycin as a treatment of COVID-19 – IHU. Library Catalog: [www.mediterranee-infection.com](http://www.mediterranee-infection.com). 4. [4]. Philippe Gautret, Jean Christophe Lagier, Philippe Parola, Van Thuan Hoang, Line Medded, Morgan Mailhe, Barbara Doudier, Johan Courjon, Valerie Giordanengo, Vera ESTEVES Vieira, Herve TISSOT Dupont, Stephane Honore, Philippe Colson, Eric Chabriere, Bernard LA Scola, Jean Marc Rolain, Philippe Brouqui, and Didier Raoult. Hydroxychloroquine and Azithromycin as a treatment of COVID-19: preliminary results of an open-label non-randomized clinical trial. medRxiv, page doi:2020.03.16.20037135, March 2020. Publisher: Cold Spring Harbor Laboratory Press. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=2020.03.16.20037135&link_type=DOI) 5. [5]. Ian R White, Nicholas J Horton, James Carpenter, Stuart J Pocock, et al. Strategy for intention to treat analysis in randomised trials with missing outcome data. Bmj, 342:d40, 2011. [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE1OiIzNDIvZmViMDdfMS9kNDAiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMC8wNC8xMC8yMDIwLjAzLjIyLjIwMDQwOTQ5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 6. [6]. Sandeep K Gupta. Intention-to-treat concept: a review. Perspectives in clinical research, 2(3):109, 2011. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.4103/2229-3485.83221&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21897887&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F04%2F10%2F2020.03.22.20040949.atom) 7. [7]. Shinichi Nakagawa and Innes C Cuthill. Effect size, confidence interval and statistical significance: a practical guide for biologists. Biological reviews, 82(4):591–605, 2007. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1469-185X.2007.00027.x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17944619&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F04%2F10%2F2020.03.22.20040949.atom) 8. [8]. Tue Tjur. Coefficients of determination in logistic regression models—a new proposal: The coefficient of discrimination. The American Statistician, 63(4):366–372, 2009. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1198/tast.2009.08210&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000271795500010&link_type=ISI) 9. [9]. Louise-Anne McNutt, Chuntao Wu, Xiaonan Xue, and Jean Paul Hafner. Estimating the relative risk in cohort studies and clinical trials of common outcomes. American journal of epidemiology, 157(10):940–943, 2003. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/aje/kwg074&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12746247&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F04%2F10%2F2020.03.22.20040949.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000182958100011&link_type=ISI) 10. [10]. Patrick Royston and Mahesh KB Parmar. Flexible parametric proportional-hazards and proportional-odds models for censored survival data, with application to prognostic modelling and estimation of treatment effects. Statistics in medicine, 21(15):2175–2197, 2002. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.1203&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12210632&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F04%2F10%2F2020.03.22.20040949.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000177051300004&link_type=ISI) 11. [11]. Joseph Coveney. Firthlogit: Stata module to calculate bias reduction in logistic regression. 2015. 12. [12]. Caitlin Rivers, Jean-Paul Chretien, Steven Riley, Julie A Pavlin, Alexandra Woodward, David Brett-Major, Irina Maljkovic Berry, Lindsay Morton, Richard G Jarman, Matthew Biggerstaff, et al. Using “outbreak science” to strengthen the use of models during epidemics. Nature Communications, 10(1):1–3, 2019. 13. [13]. Theo Lorenc and Kathryn Oliver. Adverse effects of public health interventions: a conceptual framework. J Epidemiol Community Health, 68(3):288–290, 2014. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiamVjaCI7czo1OiJyZXNpZCI7czo4OiI2OC8zLzI4OCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIwLzA0LzEwLzIwMjAuMDMuMjIuMjAwNDA5NDkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 14. [14]. Michael A Johansson, Nicholas G Reich, Lauren Ancel Meyers, and Marc Lipsitch. Preprints: An underutilized mechanism to accelerate outbreak science. PLoS medicine, 15(4), 2018. [1]: /embed/inline-graphic-1.gif