PT  - JOURNAL ARTICLE
AU  - Yin, Guosheng
AU  - Zhang, Chenyang
AU  - Jin, Huaqing
TI  - Statistical Issues and Lessons Learned from COVID-19 Clinical Trials with Lopinavir-Ritonavir and Remdesivir
AID  - 10.1101/2020.06.17.20133702
DP  - 2020 Jan 01
TA  - medRxiv
PG  - 2020.06.17.20133702
4099  - http://medrxiv.org/content/early/2020/06/19/2020.06.17.20133702.short
4100  - http://medrxiv.org/content/early/2020/06/19/2020.06.17.20133702.full
AB  - Background Since the outbreak of the novel coronavirus disease 2019 (COVID-19) in December 2019, it has rapidly spread in more than 200 countries or territories with over 8 million confirmed cases and 440,000 deaths by June 17, 2020. Recently, three randomized clinical trials on COVID-19 treatments were completed, one for lopinavir-ritonavir and two for remdesivir. One trial reported that remdesivir was superior to placebo in shortening the time to recovery, while the other two showed no benefit of the treatment under investigation. However, several statistical issues in the original design and analysis of the three trials are identified, which might shed doubts on their findings and the conclusions should be evaluated with cautions.Objective From statistical perspectives, we identify several issues in the design and analysis of three COVID-19 trials and reanalyze the data from the cumulative incidence curves in the three trials using more appropriate statistical methods.Methods The lopinavir-ritonavir trial enrolled 39 additional patients due to insignificant results after the sample size reached the planned number, which led to inflation of the type I error rate. The remdesivir trial of Wang et al. failed to reach the planned sample size due to a lack of eligible patients, while the bootstrap method was used to predict the quantity of clinical interest conditionally and unconditionally if the trial had continued to reach the originally planned sample size. Moreover, we used a terminal (or cure) rate model and a model-free metric known as the restricted mean survival time or the restricted mean time to improvement (RMTI) in this context to analyze the reconstructed data due to the existence of death as competing risk and a terminal event. The remdesivir trial of Beigel et al. reported the median recovery time of the remdesivir and placebo groups and the rate ratio for recovery, while both quantities depend on a particular time point representing local information. We reanalyzed the data to report other percentiles of the time to recovery and adopted the bootstrap method and permutation test to construct the confidence intervals as well as the P values. The restricted mean time to recovery (RMTR) was also computed as a global and robust measure for efficacy.Results For the lopinavir-ritonavir trial, with the increase of sample size from 160 to 199, the type I error rate was inflated from 0.05 to 0.071. The difference of terminal rates was −8.74% (95% CI [-21.04, 3.55]; P=.16) and the hazards ratio (HR) adjusted for terminal rates was 1.05 (95% CI [0.78, 1.42]; P=.74), indicating no significant difference. The difference of RMTIs between the two groups evaluated at day 28 was −1.67 days (95% CI [-3.62, 0.28]; P=.09) in favor of lopinavir-ritonavir but not statistically significant. For the remdesivir trial of Wang et al., the difference of terminal rates was −0.89% (95% CI [-2.84, 1.06]; P=.19) and the HR adjusted for terminal rates was 0.92 (95% CI [0.63, 1.35]; P=.67). The difference of RMTIs at day 28 was −0.89 day (95% CI [-2.84, 1.06]; P=.37). The planned sample size was 453, yet only 236 patients were enrolled. The conditional prediction shows that the HR estimates would reach statistical significance if the target sample size had been maintained, and both conditional and unconditional prediction delivered significant HR results if the trial had continued to double the target sample size. For the remdesivir trial of Beigel et al., the difference of RMTRs between the remdesivir and placebo groups up to day 30 was −2.7 days (95% CI [-4.0, −1.2]; P<.001), confirming the superiority of remdesivir. The difference in recovery time at the 25th percentile (95% CI [-3, 0]; P=.65) was insignificant, while the differences manifested to be statistically significant at larger percentiles.Conclusions Based on the statistical issues and lessons learned from the recent three clinical trials on COVID-19 treatments, we suggest more appropriate approaches for the design and analysis for ongoing and future COVID-19 trials.Competing Interest StatementThe authors have declared no competing interest.Funding StatementNo funding to disclose.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:IRB exemption category 4 - Existing dataAll necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesThe authors confirm that the data supporting the findings of this study are available within the article.