Mortality rates in matched cohort, pseudo-randomised and randomised trials of convalescent plasma given to COVID-19 patients

Introduction

Since the first reports of a cure for terminal children with Diphtheria induced laryngeal obstruction with immune serum in 1898 (1) and the successes in 1917-18 flu epidemic with use of convalescent serum (2), this approach has been successfully used in bacterial diseases such as meningitis (3) and viral epidemics (4) most recently including Ebola (5) Unfortunately to date, most of the success has depended on case-control studies (6) rather than randomised trials to properly establish if there is a real benefit from this approach. The first report of its potential benefit compared to matched untreated patients in severe COVID-19 infection was in April 2020 (7). Since then there has been increasing interest (8, 9) and three ongoing Cochrane reviews which have demonstrated that un-matched cohort studies are still the most frequent reports (10-14). The most recent Cochrane report identified 2 underpowered randomised control trials and 8 studies with matched controls, though because of identified bias being high in the matched studies they only assessed mortality in the randomised trials, which though reduced was not significant due to lack of numbers.

Methods

Searches of Pubmed, EMBASE, Web of Science and ClinicalTrials.gov have been undertaken using the following terms: Convalescent Plasma and COVID-19 and year published 2020. At the last search 22.10.2020, 153 references were identified through Medline 143 through EMBASE, 363 through Web of Science and 96 from ClinicalTrials.gov currently recruiting and 16 completed. To be included the title or abstract had to indicate that their data related to a randomised, pseudorandomised (see below for discussion of rationale for inclusion) or Matched Cohort study. To compared the results with data on established COVID-19 therapies, three randomised trials with Remdesivir (15-17) and one trial with Dexamethasone (18) were reviewed

From the review, a total of 16 papers with identified original data comparing convalescent plasma and randomised (n=3), pseudo-randomised (n=2) or matched cohort controls (n=11) were read as full papers. A seventeenth paper (19) was identified through the review of Joyner (8). The original article gave no indication of number of controls (they did indicate in the manuscript that the results compared favourably with their institutional database). For this paper, as the review by Joyner documented such data but without information on follow up, this information has been used in this review.

The decision for exploring the data from pseudo-randomised series was because If a treatment is not randomised but is given “in a perceived” uniform cohort manner but post-treatment completion, a significant prognostic factor unsuspected at the trial’s initiation emerges as a risk factor that influencing survival, this factor could not be deemed to have influenced actual allocation. In such a case, Joyner et al (https://pubmed.ncbi.nlm.nih.gov/32817978/) considered those allocated by chance to the extremes of the perceived specific new risk factor, are considered to be pseudo-randomised and therefore of possibly of more value for examining this role of this specific factor than a conventional matched cohort study though still not as good as specific randomisation.

If two such significant prognostic factors unsuspected at the trial’s initiation emerges as separate risk factors that influencing survival and prove synergistic in their interaction, could the outcome of double positives versus double negatives in such a “Pseudo-pseudo” Randomisation of two prognostic factors provide a closer approximation to true randomisation than Pseudo-randomisation? In using two prognostic factors it is following the concept of Mendelian randomisation except it uses prognostic factors instead of gene definitions. Of course, it still does not reach the reliability of a randomised controlled trial, but the composite control double negatives could be closer to a true placebo effect than a randomised trial without a precise placebo because of no way of determining the impact of the placebo effect.

In the largest “pseudo-randomised” study (20), those receiving the lowest and highest titre plasma were compared. In the smaller study (21), the comparison was an arbitrary cut-point based on the levels of serum IgG against the spike protein of SARS-Cov-2 measured by an enzyme-linked immunosorbent assay (ELISA) using a validated commercial kit (Euroimmun AG, Luebeck, Germany, product number EI 2606-9601G) comparing the upper 39% versus the lower 61%. Two Chinese studies (22, 23) were pooled as the first had pooled small numbers of three early reports with 19 patients with 10 matched controls and the latter reported on the formal phase II study which followed.

For each study, the proportion that died and follow up after treatment and control groups were tabulated and absolute and relative risk calculated. The same information was obtained from three Remdesivir and one Dexamethasone trials all randomised to compare with the outcome from convalescent plasma data.

Results

A total of 2,378 Convalescent plasma (CP) treated and 5188 “controls” were identified in the 17 studies and for the analysis, the studies of Chan et al (30) and Xie et al (23) were merged making 16 comparisons (Table 1). This table also indicated number and percentage of deaths and the duration of follow up which varied from >5 to 30 days (median 17.5 days). Absolute reductions were calculated from the difference between percentage deaths in Controls vs CP groups and ranged from 2.4% to 23.9% (median 11.5%) for the 11 matched cohort comparisons while it was –0.9%, 8.3% and 11.6% for the randomised trials and 7.3% and 12.8% for the pseudorandomised studies. The overall absolute risk reduction was median 10.8%, mean 11.72%. Using the difference between the median frequency of deaths in the treated (13.4%) versus controls (26.7%), the sample size of a randomises trial needed to treat to prove this difference would be 141 in each arm while in the worst-case scenario 1.9% versus 4.3% it would be 817 per arm.

The most statistically significant study was the first pseudo-randomised study with 515 high titre vs 561 low dose titre plasma (Table 1, 22.3% vs 29.6% deaths after 30 days follow up with RR = 0.755 (95% CI: 0.614, 0.927), z-value = −3.528 and p= 0.007). This was even larger when the difference in time to transfusion, =<3 days verses > 4days of COVID-19 diagnosis was also taken into account in a further “pseudo-pseudo” randomisation (Table 2 16.7% in 180 patients vs 31.8% death rate in 371 patients at 30 days RR = 0.524 (95% CI: 0.366, 0.750) p= 0.0004).

Despite only one other study being statistically significant individually, all of the smaller studies had approximately a similar percentage risk reduction (Table 1).

We performed four random meta-analyses using the restricted maximum likelihood method estimator. The pooled estimate of all study types (i.e. matched cohort studies, randomised trials and pseudo randomised trials) shows a statistically significant reduction in COVID-19 deaths with a RR of 0.713 (95% CI: 0.601, 0.846), z-value = −3.880 and p-value = 0.0001. Similarly, a statistically significant reduction in COVID-19 deaths from the eleven matched cohort studies meta-analysis with a RR of 0.610 (95% CI: 0.459, 0.810), z-value = −3. 418 and p-values = 0.001. However, the result of the meta-analysis of the three randomised trials did not show a statistically significant reduction of COVID-19 deaths RR = 0.825 (95% CI: 0.529, 1.287), z-value = −0.846 and p-value = 0.397. When the two pseudo-randomised studies were pooled together, there were a statistically significant reduction in COVID-19 deaths RR = 0.747 (95% CI: 0.610, 0.916), z-value = −2.807 and p-value = 0.005.

The result of the meta-regression mixed-effects model estimated no statistically significant differences between the three study types. The estimated effect of the pseudo-randomised studies compared with the matched cohort studies was 0.141 (95% CI: −0.442, 0.724) with z-value = 0.473 and p-value = 0.636. Similarly, no statistically significant difference was observed between the randomised trials and the reference group the matched cohort studies 0.267 (95% CI: −0.291, 0.825), z-value = 0. 938 and p-value = 0.348. The estimated I² = 40.40% indicating non-statistically significant moderate observed heterogeneity with p-value = 0.092 of the test statistic, which examines the null hypothesis that all studies are evaluating the same effect.

A comparison of the mortality rate from convalescent plasma studies and the currently licenced products Dexamethasone and Remdesivir is shown in Table 3 and a comparison of recovery times of convalescent plasma vs Remdesivir and the monoclonal antibody Regeneron is shown in Table 4

Two studies investigated the impact of Convalescent Plasma on persistence of COVID-19 Viral nucleic acid detection. Both showed faster decline compared to control but only one showed a statistically significant effect at 72 hours. (Table 5).

Discussion

With more than a million COVID-19 deaths world wide (https://www.ecdc.europa.eu/en/geographical-distribution-2019-ncov-cases) and more than 6,000 a day, there is a need for more than a vaccine to control this pandemic. While agreeing with recent editorials calling for more effort to recruit to randomised trials with Convalescent Plasma (9, 31), it is clear from the figures in Table 1 (only 329 of 2378 – 13.8% receiving convalescent plasma in a randomised trial) that, faced with the initial high mortality rate, most clinicians actually treating patients preferred first to demonstrate benefit compared to untreated but matched controls. By no means, without results from larger series of randomised trials, can the mixed standards of reporting in this systematic review be trusted on their own. Some studies had unknown antibody titre, there was differences in length of follow-up and confounding by co-administration of other COVID-19 treatments in the Matched cohort series. They are certainly all biased to a degree as indicated by those included in the Cochrane review which included series up to the beginning of August (11). However, in the absence of any better material, accepting these imperfections, from the meta-analyses of death rates there is a highly significant reduction of death in the period up to 30 days after transfusion (Table 3 RR= 0.713,, Z= −3.680 p< 0.0001) Furthermore, this is not far from the non-significant findings reported in two of the under-powered randomised studies (RR 0.654 and 0.545) though it is better than the Indian randomised trial reported by Agarwal et al Table 1 (RR 1.069 (32)). This trial used young donors who only had mild symptoms and 72% of recipients received plasma with a low titre or no antibody when checked retrospectively.

Despite the significant reduction of deaths in the pooled analysis, the reduction in death is less than the odds ratio, 0.25; 95% confidence interval, 0.14–0.45; I² = 0% p< 0.001 reported from matched cohort studies of convalescent plasma before the COVID-19 pandemic (4). However, like these studies, the COVID-19 randomised data from Li et al (33) and “pseudo-randomised” data from Joyner et al (20) do suggest greater benefit from treating less extreme cases, treating within 3 days of onset of illness and use of high antibody titre sera. The current results are certainly better when compared to those reported in both the dexamethasone trial and the Remdesivir trials (15, 17, 34) (Table 3). This is most critically emphasised in the study of Wang et al (15) which showed absolutely no difference in the rate of decline of Viral load following Remdesivir than in the control group over 28 days. As table 5 demonstrates Li et al (33) demonstrated a markedly different effect from the study of convalescent plasma when significant reductions are apparent within 24 hours and 87.2% of CP treated compared to 31.8% in the randomised control group being negative within 72 hours (33). This advantage was present both in early severe disease and advanced life-threatening disease reaching p<.001 in both groups by 72 hours. Despite this similar early effect, the latter group had slower recovery and did not show survival advantage at 30 days follow up possibly because of more systemic damage. The study of viral disappearance was markedly less impressive in Agarwal et al’s trial (32). This was the only trial showing no reduction of mortality. A possible explanation could have been the low rate of high titre anti-COVID-19 antibody in their donors (26.7% had no detectable antibody and only 28.0% had titres >1:80)

Critical to the justification of pursuing randomised trials is the need to confirm that the complications of convalescent plasma are as low as reported (11, 35) particularly given the anxiety from animal studies about immune enhancement (36) and serum screening of COVID-19 survivors have detected autoantibodies to Interferon (37). This will be important to rule out, given that Convalescent plasma is much less toxic than those seen in the randomised trials supporting the two licenced drugs. These showed possible (though non-significant) harm to patients not receiving respiratory support while undergoing Dexamethasone (18) and 21% of patients in the 5-day group and 35% in the 10-day group having serious adverse events while undergoing Remdesivir treatment (38).

Hopefully, this report will help accelerate completion of more than 4 randomised trials involving nearly 4,000 patients. These were expected within the next 1-3 months but recruitment slowed due to a Summer decline in advanced cases in the northern hemisphere (39) and the need to select only those donors with high titre neutralising antibodies against the SARS-CoV-2 spike protein. It is hoped that these can be completed soon as trials with the high titre monoclonal antibodies (40) are now beginning (41, 42). Though no data has been published in a scientific journal, a press release of early results from treating COVID-19 antibody-negative non-hospitalised patients are indicating activity (https://investor.regeneron.com/news-releases/news-release-details/regenerons-regn-cov2-antibody-cocktail-reduced-viral-levels-and/). The only possible direct comparison between these early results and this report is when the results of time to recovery are examined. After the monoclonal antibody treatment versus untreated controls there is 7 vs 13 days, 46.2% reduction. This compares to 11 vs 15 days (26.7% reduction) after Remdesivir (34) and 6.25 vs 12.88 days, (51.5% reduction in a CP trial (43) (Table 4). Completing the Convalescent plasma trials will also be important for future epidemics as convalescent sera can become available more quickly than monoclonal antibodies. This is particularly important for areas without access to western-style medicine (44). Equally, there could be something in recovered plasma which may be related to other determinants on the virus that contributes to virulence than the monoclonal antibody recognises. This means that it will be important that monoclonals are tested against CP in randomised trials. The most sensitive way of testing these two approaches could be a 2×2 randomised trial of high titre CP versus Monoclonal and a second randomisation for immediate versus deferred use based on the rate of decline in Virus titre over 72 hours.

Missing information from the currently available published data is the impact of convalescent plasma in patients with extreme poor risk factors such as obesity, diabetes, old age and ethnicity. In the US EAP COVID-19 consortium study, there is double the death rate in over 70s though just under 2/3rds were alive after 30 days verses 4/5^th of the under 70s (20). In addition, there has so far been no study providing long term survival and quality of life data in CP patients, given the increasing recognition of the problem of late persistent poor quality of life in a significant minority of recovered patients (45).

The final issue worth further investigation is the impact of Vitamin D deficiency, as there have been no studies of the influence of Vitamin deficiency on failure to respond to Convalescent plasma. In the last month, there have been increasing interest (46) with six reports demonstrating a strong impact of Vitamin D deficiency on the behaviour of SARS-CoV-2 infection. Firstly, Meltzer et al studied Vitamin D levels in 489 attendees in an urban medical centre in Chicago (47). They demonstrated a higher frequency of acquisition of COVID-19 acquisition in the Vitamin D deficient group (21.9%) than in the sufficient group (12.2%). This was subsequently confirmed by a much larger study (48) and two smaller studies (46, 49). The Boston group also reported data suggesting a link between Vitamin D deficiency and increased severity of disease (50). This might explain the failure to demonstrate any short term survival benefit of convalescent plasma in patients with life-threatening disease in the randomised trial of Li et al despite equal efficiency in dealing with actual presence of the virus within 72 hours of transfusion (Table 5). It might be important if this prediction proves to be true, as patients with life-threatening disease could respond better to treatment with more rapidly acting Calcifediol rather than Vitamin D3 given the recent report from Spain (51). These authors reported a small randomised study which demonstrated that only 1 of 50 hospitalised COVID-19 patients receiving Colifediol required admission to intensive care unit versus 13 of 26 untreated controls (p< 0.001)

Conclusion

It is 10 months since this pandemic and worldwide there continue to be close to 6,000 deaths a day and total of more than 1 million deaths

The data analysed in this paper suggests CP given within 3 days of admission at an adequate titre could reduce deaths by as much as 40% and is better than that achieved from any other licenced drug, with less side effects

Expansion of current randomised trials and more education amongst potential donors and the medical profession in general to enable more widespread access to such trials are required as we enter the second wave of this pandemic.

Conflicts of Interest

None

Authors Contribution

each author contributed equally, MS & TO primarily extracting data and AA undertaking statistical analysis

Funding

There was no funding