Abstract
Misconceptions and ill-founded theories can arise in all areas of science. However, the apparent accessibility of many epidemiology findings and popular interest in the subject can lead to additional misunderstandings. The article below is the third in an occasional series of short editorials highlighting some current misinterpretations of epidemiological findings. Invited authors will be given wide scope in judging the prevalence of the misconception under discussion. We hope that this series will prove instructive to cancer researchers in other disciplines as well as to students of epidemiology. Adrian L Harris and Leo Kinlen
Main
For breast cancer, as for many other diseases, treatment that is appropriate for one patient may be inappropriate for another. Ideally, therefore, what is wanted from trials is not only an answer to the question ‘Is this treatment helpful on average for a wide range of patients?’, but also an answer to the question ‘For which recognisable categories of patient is this treatment particularly helpful?’ (EBCTCG, 2005a,2005b)
In general, however, this ideal cannot be achieved directly from subgroup-specific analyses of clinical trial results because apparent differences between the proportional risk reductions in different subgroups of the patients in a trial (or even in a meta-analysis of many trials) are often surprisingly unreliable. For example, even if the proportional effects of the trial treatment in specific subgroups really are importantly different, standard subgroup analyses are so insensitive that they may well fail to demonstrate these real differences. Conversely, even if the trial results suggest that the trial treatment works in some subgroups but not in others (thereby giving the appearance of a ‘qualitative interaction’), this may still not be good evidence for subgroup-specific treatment preferences. The play of chance often produces qualitatively wrong answers in particular subgroups in trials (or in meta-analyses of trials) that could, if interpreted incautiously, lead to millions of people being treated inappropriately or untreated inappropriately.
Questions about such ‘interactions’ between patient characteristics and the effects of treatment are easy to ask, but are surprisingly difficult to answer reliably. Apparent interactions can often be produced by the play of chance and, in particular subgroups, can mimic or obscure some of the moderate treatment effects that might realistically be expected. To illustrate this, a subgroup analysis was performed based on the astrological birth signs of 17 000 patients in the Second International Study of Infarct Survival (ISIS-2), a randomised trial of 1 month of daily aspirin vs placebo for suspected acute myocardial infarction. Overall in this trial, the 1-month survival advantage produced by aspirin was demonstrated conclusively (804 vascular deaths among 8587 patients allocated aspirin vs 1016 among 8600 allocated no aspirin; 23% proportional reduction, P<0.000001). To demonstrate the unreliability of subgroup analyses, these findings were subdivided into 12 subgroups according to the patients’ medieval astrological birth signs; the results in each were examined to find which of the 12 appeared least promising, and when just those with apparently unpromising results were collected together, it was ‘discovered’ that aspirin appeared totally ineffective for patients born under Libra or Gemini (Table 1) (ISIS, 1988)!
It would be unwise to conclude from such a result that patients born under the astrological birth sign of Libra or Gemini should not be given aspirin if they have a heart attack. However, similar conclusions based on ‘exploratory’ data-derived subgroup analyses, which from a purely statistical viewpoint are no more reliable than these astrological subgroup analyses, are often reported and believed, with inappropriate effects on worldwide clinical practice.
There are three partial remedies for this unavoidable conflict between the reliable subgroup-specific conclusions that doctors and patients want and need, and the statistically unreliable findings that direct subgroup-specific analyses can usually offer. However, the extent to which these remedies are helpful in particular instances is one on which informed judgements differ.
First, where there are good prior reasons for anticipating that the proportional effects of treatment might be very different in different circumstances, one particular subgroup analysis may be prespecified in the study protocol, along with a prediction of the direction of the proposed interaction. (For example, it was expected that the benefits of fibrinolytic therapy for acute myocardial infarction would be greater the earlier such patients were treated, and so some studies prespecified that the statistical analyses would be subdivided by the number of hours from the onset of symptoms to treatment: FTT, 1994.) Although a single prespecified subgroup-specific analysis can then be taken somewhat more seriously than other subgroup analyses, protocols that pre-specify several subgroup analyses as ‘secondary outcomes’ can yield importantly wrong answers.
The second approach is to take the proportional risk reduction that is suggested by the overall results of the trial (or, better still, by the overall results from a meta-analysis of all such trials) as a semi-quantitative guide to the proportional risk reductions in various specific subgroups of patients, giving little weight to the apparent results in each of such subgroups. This is clearly the right way to interpret the astrological ‘findings’ in Table 1, and, if used sensibly, may also in many other circumstances provide the best guide as to whether one treatment is better than another in particular subgroups.
The main determinant of whether toxic or expensive treatment is worthwhile is the absolute risk reduction that it produces, and it is perfectly proper to use the fact that patients who already have a very good prognosis anyway and are at low absolute risk cannot have a large absolute benefit (for, even if a small risk is halved the absolute benefit is small). Classification of patients as being at low (or high) risk of an adverse outcome is often a useful guide as to which patients can expect a small (or large) absolute gain. Appropriate clinical use of this low-risk/high-risk split may not require support from formal subgroup analyses – indeed, it could even be damaged by incautious reliance on such analyses.
The third approach is to be influenced, in discussing the likely effects on mortality in specific subgroups of breast cancer patients, not only by mortality analyses but also by analyses of recurrence, early recurrence, local recurrence or some other major ‘surrogate’ outcome. For, if the overall results are similar but much more highly significant for recurrence than for mortality, subgroup analyses with respect to the former may be more stable and may provide a better guide as to whether there are any major differences between subgroups in the proportional risk reduction produced by treatment (EBCTCG, 2005a,2005b).
The appropriate interpretation of apparently different results in different subgroups of trial results is still one of the most difficult matters of judgement in the interpretation of randomised evidence; at present, many clinicians and regulatory agencies pay far too much attention to irregularities between the apparent effects in different subgroups, to the potential detriment of the care of individual patients.
Change history
29 March 2012
This paper was modified 12 months after initial publication to switch to Creative Commons licence terms, as noted at publication
References
Early Breast Cancer Trialists’ Collaborative Group (EBCTCG) (2005a) Effects of chemotherapy and hormonal therapy for early breast cancer on recurrence and 15-year survival: an overview of the randomised trials. Lancet 365: 1687–1717
Early Breast Cancer Trialists’ Collaborative Group (EBCTCG) (2005b) Effects of radiotherapy and of differences in the extent of surgery for early breast cancer on local recurrence and 15-year survival: an overview of the randomised trials. Lancet 366: 2087–2106
Fibrinolytic Therapy Trialists’ Collaborative Group (FTT) (1994) Indications for fibrinolytic therapy in suspected acute myocardial infarction: collaborative overview of early mortality and major morbidity results from all randomised trials of more than 1000 patients. Lancet 343: 311–322
ISIS-2 (Second International Study of Infarct Survival) Collaborative Group (1988) Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17 187 cases of suspected acute myocardial infarction: ISIS-2. Lancet 332: 349–360
(AND FURTHER READING ON TRIAL INTERPRETATION)
Baigent C, Peto R, Gray R, Parish S, Collins R . Large-scale randomized evidence: trials and meta-analyses of trials. (pp 31–45 in the electronic version of Warrell DA, Cox TM, Firth JD (eds)). Oxford Textbook of Medicine 5th edn. Oxford University Press: Oxford, 2010
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
From twelve months after its original publication, this work is licensed under the Creative Commons Attribution-NonCommercial-Share Alike 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/
About this article
Cite this article
Peto, R. Current misconception 3: that subgroup-specific trial mortality results often provide a good basis for individualising patient care. Br J Cancer 104, 1057–1058 (2011). https://doi.org/10.1038/bjc.2011.79
Published:
Issue Date:
DOI: https://doi.org/10.1038/bjc.2011.79
This article is cited by
-
Diabetes precision medicine: plenty of potential, pitfalls and perils but not yet ready for prime time
Diabetologia (2022)
-
Remdesivir for COVID-19 pneumonia: still undecided, but it might all be about adequate timing
European Journal of Clinical Pharmacology (2021)
-
OptiBIRTH: a cluster randomised trial of a complex intervention to increase vaginal birth after caesarean section
BMC Pregnancy and Childbirth (2020)
-
Causal associations of blood lipids with risk of ischemic stroke and intracerebral hemorrhage in Chinese adults
Nature Medicine (2019)
-
Lipid management in patients with chronic kidney disease
Nature Reviews Nephrology (2018)