Molnupiravir’s authorisation was premature

Dangers of truncated trials

Move-Out was stopped prematurely for reasons that aren’t completely clear. When the trial was published, the authors reported a smaller absolute difference in primary outcome (−3%, 95% confidence interval −5.9 to −0.1, P=0.043)7 than quoted in Merck’s earlier press release (−6.8%, P=0.0012). As the upper limit of this 95% confidence interval approaches zero, these published results have borderline significance, indicating that even a small number of misclassified outcomes could overturn the significance of the findings.8 Robustness is further undermined by a modified intention-to-treat analysis that excluded 25 patients after randomisation. Whether a standard intention-to-treat analysis of all randomised patients would have produced significant results remains unclear.

The weakness of the evidence supporting molnupiravir has been partially obscured by the large relative benefit reported in Merck’s original press release. This created a cognitive bias by anchoring public and scientific opinion to a belief in substantial benefit. Press releases can also set unreasonable expectations among the public and policy makers, leading to demand for immediate access to poorly evaluated drugs.

Good evidence exists that prematurely terminated trials are more likely than non-truncated trials to overestimate effect sizes. In one meta-analysis, 91 truncated trials reported greater effect sizes than 424 matched non-truncated trials,9 independent of statistical stopping rules. Smaller truncated trials with less than 200 events (Move-Out had 112 events) reported a 63% greater reduction in relative risk than similar trials that were not truncated. Interestingly, Move-Out’s truncated relative risk reduction (0.5)1 is 66% higher than the non-truncated relative risk reduction (0.3),⁷ emphasising the dangers of making decisions based on a single prematurely terminated trial.

Clinical value?

The real question for any trial is whether the findings are clinically as well as statistically significant. There are two main challenges in answering that question: firstly, determining how best to define clinical significance and, secondly, the fact that null hypothesis significance testing is inadequate for evaluating the clinical relevance of an effect. As Move-Out was powered to detect an absolute reduction of 6% in hospital admissions or death, we might reasonably accept this as the minimum clinically meaningful difference. The confidence interval around the published reduction in primary outcome (−3%, 95% CI −5.9 to −0.1)7 excludes 6%, leading to questions about the clinical importance of these findings.

Molnupiravir’s safety profile is also uncertain because Move-Out was underpowered, like most clinical trials, to detect clinically important side effects. Its mutagenic mode of action could in theory encourage emergence of additional SARS-CoV-2 variants. The generalisability of the results is also unclear as the trial population was unvaccinated and recruited before the most recent variants emerged, with 70% from low and middle income countries. It is not known whether reported benefits will be maintained across different populations with differing variants, ancillary treatments, and healthcare systems. Further uncertainty arises from two more recent studies of molnupiravir reporting no clinical benefit in either outpatients or inpatients with covid-19.1011

The importance of clinical benefit is highlighted by molnupiravir’s high costs. Five days’ treatment is expected to cost about $700 (£500; €600), and with the manufacturer estimating that demand will reach 10 million courses over the next year,1 global costs could reach several billion dollars annually. The opportunity costs associated with this agent, including the potential for more financial support for mass vaccinations in low and middle income countries, merit serious reflection.

Detailed cost effectiveness studies by independent health technology assessment groups must be done to avoid repeating past mistakes. Approval of oseltamivir (Tamiflu), an antiviral for early uncomplicated influenza, resulted in worldwide governmental spending exceeding $18bn (half on stockpiling) for a drug with no benefit other than reducing duration of symptoms by less than one day.12

The evidence for incorporating molnupiravir into routine practice is fragile. Premature regulatory authorisation and guideline recommendation¹³ on the basis of truncated and non-replicated trial findings and without full consideration of clinical, not statistical, significance and cost effectiveness falls far short of the wise stewardship of limited healthcare resources required during a global healthcare emergency. We deserve and should demand better.