ReviewIncreasing value and reducing waste in biomedical research: who's listening?
Introduction
More than 30 years ago, the adverse clinical consequences of biased under-reporting of research were clearly documented1 and non-publication of research remains hugely problematic.2, 3, 4, 5 Non-publication is bad value for funders, who could double research output by ensuring all the funded studies are published, and this situation puts patients and clinicians at a substantial disadvantage in making informed decisions about health care.6 Trial registration, supported by the International Committee of Medical Journal Editors (ICMJE),7 has helped to address this problem8, 9 although this solution is clearly not a panacea.10, 11 Other related initiatives, such as the Alltrials initiative and the Institute of Medicine's report on data sharing12 are working to ensure that the results of all trials are reported and that their data are made available.
Chalmers and Glasziou13 estimated in 2009 that 85% of research funding was being avoidably wasted across the entire biomedical research range (eg, clinical, health services, and basic science). Evidence of the extent and avoidability of waste in research production at each stage of the authors' four stage model has grown, and has confirmed imbalanced research question selection,14 poor study design15, 16 and execution, non-publication,17 and poor reporting18 and some have suggested that a more fundamental reassessment is needed in how research priorities are developed and pursued.19, 20 In addition to 228 citations as of Sept 11, 2015, the 2009 paper13 led National Institute of Health Research (NIHR) in England to establish a working group to monitor and plan actions, with regular meetings and an annual closed conference. The NIHR's Adding Value in Research programme had an additional stage (figure 1) aiming to ensure that NIHR funded research: addresses questions relevant to clinicians, patients, and the public; uses appropriate design and methods; is delivered efficiently; results in accessible full publication; and produces unbiased and useable reports. NIHR developed a quality improvement method21 for these five stages to identify common themes and examples of good practice across their programmes. For example, since 2013, NIHR has required applicants for support of new primary research to reference an existing systematic review “as well as including reference to any relevant literature published subsequent to that systematic review” or when no such systematic review exists, applicants should review the relevant evidence (with a method that systematically identifies, critically appraises, and combines the evidence), which “must also include reference to relevant on-going studies, eg, from trial registries”.22
In 2014 The Lancet published a Series (“Increasing value: reducing waste”)23, 24, 25, 26, 27 extending the 2009 analysis from 4 to 50 journal pages, with more than 40 authors focused on the five NIHR stages. As the Commissioning Editors noted: “Our belief is that research funders, scientific societies, school and university teachers, professional medical associations, and scientific publishers (and their editors) can use this Series as an opportunity to examine more forensically why they are doing what they do…and whether they are getting the most value for the time and money invested in science.”28
The Series, and an accompanying symposium,29 provided a voluminous body of evidence for the issues in biomedical research, along with 17 recommendations (table 1) to help to increase value, covering funders, regulators, journals, academic institutions, and researchers. These issues include (although they are not limited to) whether planned research met the needs of end users.30, 31, 32
Initial media attention included coverage by several newspapers including the leading German paper, Der Spiegel,33 although almost no response has been made from German researchers or organisations (Antes G, German Cochrane Centre, personal communication). Several research funders responded through meetings, working parties, and some changes of processes. In the year since their publication, the five articles have been downloaded 46 596 times from The Lancet and ScienceDirect websites. The five Series papers have already been cited 113 times (Scopus); were all in the top 5% of all articles indexed by Scopus; and their alternative metric scores (used to measure social media) all ranked more than the 98th percentile (of more than 3 million articles scored) including 589 tweets (about 20% of which were by health-care professionals).
This follow-up Review offers an overview of the initial stimulus of the Series. Before the assessment, a protocol was developed outlining the key players and the methods of our investigation, including sampling frames (panel 1). The primary focus was to assess what funders, regulators, journals, academic institutions, and researchers are doing, and plan to do, to address waste in biomedical research.
Section snippets
Funders
A few funders have already responded to the Series. In May, 2014, the French Institute of Health and Medical Research, Institut National de la Santé et de la Recherche Médicale (INSERM), in conjunction with the Enhancing the Quality and Transparency of Health Research (EQUATOR) network, organised a 1-day conference in Paris on “Improving reporting to decrease the waste of research” with the director of the Wellcome Trust and NIHR's Health Technology Assessment programme among the speakers
Regulators
Regulators can help with this goal by not providing ethics approval of protocols that are scientifically poor, which would mean that these protocols are also ethically inadequate. For example, the guidance for researchers issued by the newly established Health Research Authority (HRA)49 in the UK now states: “Any project should build on a review of current knowledge. Replication to check the validity of previous research is justified, but unnecessary duplication is unethical.”
Conversely,
Journals
In view of the fact that more than half of the reports of clinical trials do not set their results in the context of the totality of evidence,27 journals have much work to do to improve this situation. Journals can progress by providing specific guidance on their websites about this crucial feature and by providing similar guidance to peer reviewers. In response to the Series, The Lancet strengthened the journal's requirement to put research into context (table 1).53 From the beginning of this
Academic institutions
We are aware of very little explicit attention by academic institutions to the Lancet Series. An exception has been in Iran, where a group of academics are running a series of workshops on the Lancet series. Two workshops on “Biomedical research: increasing value, reducing waste” were run in February, 2015, for Directors of Clinical Research Centers, research vice chancellors, and Director Generals of Research Affairs of Medical Universities of North West Universities of Iran. A final national
Researchers
Motivated by the principle that initiation of research without a systematic review of already known evidence is unethical, unscientific, and wasteful, particularly when the research involves people or animals, three Scandinavian researchers66 convened and inaugurated an international Evidence-Based Research Network at the end of 2014. This network will urge funders, regulators, researchers, academic institutions, and journals to implement the changes needed to promote evidence-based research.
Looking to the future
The overall response to the 2014 Series might be summed up as some gratifying actions, but much more needs to be done to reduce research waste than at present. From a bibliometric and social media perspective, the Series has gained some traction, which is encouraging. Recognition of the problems described in the Series, and dialogue about the recommendations and possible ways to monitor progress are important first steps. However, if researchers are to avoid the well known issue of not
References (79)
- et al.
Avoidable waste in the production and reporting of research evidence
Lancet
(2009) - et al.
How to increase value and reduce waste when research priorities are set
Lancet
(2014) - et al.
Increasing value and reducing waste in research design, conduct, and analysis
Lancet
(2014) - et al.
Increasing value and reducing waste: addressing inaccessible research
Lancet
(2014) - et al.
Reducing waste from incomplete or unusable reports of biomedical research
Lancet
(2014) - et al.
How should medical science change?
Lancet
(2014) - et al.
Further emphasis on research in context
Lancet
(2014) - et al.
Elevating the quality of disability and rehabilitation research: mandatory use of the reporting guidelines
Arch Phys Med Rehabil
(2014) - et al.
A new network to promote evidence-based research
Lancet
(2014) - et al.
Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings
J Clin Epidemiol
(2005)
Impact of an online writing aid tool for writing a randomized trial report: the COBWEB (Consort-based WEB tool) randomized controlled trial
BMC Med
Study of information submitted by drug companies to licensing authorities
BMJ
Publication bias: the case for an international registry of clinical trials
J Clin Oncol
Discontinuation and non-publication of surgical randomised controlled trials: observational study
BMJ
Sharing and reporting the results of clinical trials
JAMA
Publication bias in reports of animal stroke studies leads to major overstatement of efficacy
PLoS Biol
An unfinished trip through uncertainties
BMJ
Clinical trial registration: a statement from the International Committee of Medical Journal Editors
CMAJ
Trial Registration at ClinicalTrials.gov between May and October 2005
N Engl J Med
Association between randomised trial evidence and global burden of disease: cross sectional study (Epidemiological Study of Randomized Trials–ESORT)
BMJ
Comparison of registered and published primary outcomes in randomized controlled trials
JAMA
Prevalence of primary outcome changes in clinical trials registered on ClinicalTrials.gov: a cross-sectional study
F1000 Res
Discussion framework for clinical trial data sharing: guiding principles, elements, and activities
Patients', clinicians' and the research communities' priorities for treatment research: there is an important mismatch
Res Involv Engagem
Avoidable waste of research related to inadequate methods in clinical trials
BMJ
The need for randomization in animal trials: an overview of systematic reviews
PLoS One
Fate of clinical research studies after ethical approval—follow-up of study protocols until publication
PLoS One
Reporting quality of social and psychological intervention trials: a systematic review of reporting guidelines and trial publications
PLoS One
Good Pharma: the public-health model of the Mario Negri Institute
The truth in small doses: why we're losing the war on cancer—and how to win it
Adding value in research quality improvement template
Guidance notes for applicants that ensure all primary research is informed by a review of the existing literature
Increasing value and reducing waste in biomedical research regulation and management
Lancet
Symposium on the Lancet Series on Research: Increasing value, reducing waste
The PCORI perspective on patient-centered outcomes research
JAMA
Driving up the quality and relevance of research through the use of agreed core outcomes
J Health Serv Res Policy
Deliberative dialogue on developing consensus on measurement and presentation of patient important outcomes in systematic reviews: a preconference meeting at OMERACT 12
J Rheumatol
Systemkritik: Wissenschaftselite beklagt zu viel Forschungsmüll
(AIM or “Core Clinical”) Journal Titles
Cited by (292)
A Decade of Efforts to Add Value to Child Health Research Practices
2024, Journal of PediatricsRacial, Ethnic, and Gender Diversity in United States Ophthalmology Clinical Trials
2024, Ophthalmology ScienceA cross-sectional analysis identified co-authorship networks and scientific collaboration on reporting guidelines for health research
2023, Journal of Clinical Epidemiology