RT Journal Article
SR Electronic
T1 Enough evidence and other endings: a descriptive study of stable Cochrane systematic reviews in 2019
JF medRxiv
FD Cold Spring Harbor Laboratory Press
SP 19013912
DO 10.1101/19013912
A1 Hilda Bastian
A1 Lars G. Hemkens
YR 2019
UL http://medrxiv.org/content/early/2019/12/09/19013912.abstract
AB Background From 2006 to 2019, Cochrane reviews could be designated “stable” if they were not being updated but highly likely to be current. This provides an opportunity to observe practice in ending systematic reviewing and what is regarded as enough evidence.Methods We identified Cochrane reviews designated stable in 2013 and 2019 and reasons for this designation. For those with conclusions stated to be so firm that new evidence is unlikely to change them, we assessed conclusions, strength of evidence ratings, and recommendations for further research. We assessed the fate of the 2013 stable reviews. We also estimated usage of formal analytic methods to determine when there is enough evidence in protocols for Cochrane reviews.Results Cochrane reviews were rarely designated stable. In 2019, there were 507 stable Cochrane reviews (6.6% of 7,645 non-withdrawn reviews). The most common reasons related to no, little, or infrequent research activity expected (331 of 505; 65.5%). Only 39 reviews were stable because of firm conclusions unlikely to be changed by new evidence (7.7%), but that declaration was mostly not supported by judgments made in the review about strength of evidence and implications for research. Among the 180 reviews stable in 2013, 16 reverted to normal status (8.9%), with 2 of those changing conclusions because of new studies. Few Cochrane protocols specified an analytic method for determining when there was enough evidence to stop updating the review (116 of 2,415; 4.8%).Conclusion Cochrane reviews were more likely to end because important future primary research activity was believed to be unlikely, than because there was enough evidence. Judgments about the strength of evidence and need for research were often inconsistent with the declaration that conclusions were unlikely to change. The inconsistencies underscore the need for reliable analytic methods to support decision-making about the conclusiveness of evidence.Competing Interest StatementThe authors have declared no competing interest.Funding StatementNo external funding was received. Hilda Bastian received support from an Australian Government Research Training Program Scholarship.Author DeclarationsAll relevant ethical guidelines have been followed; any necessary IRB and/or ethics committee approvals have been obtained and details of the IRB/oversight body are included in the manuscript.YesAll necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesData and supporting files, including analytic code, are available on GitHub. https://github.com/hildabast/enough-evidence CDSRCochrane Database of Systematic Reviews (journal component of The Cochrane Library)CRGCochrane review groupGRADEGrading of Recommendations, Assessment, Development and EvaluationNICENational Institute for Health and Care EvidenceSoFGRADE Summary of Findings table in Cochrane reviewsXMLeXtensible Markup Language (for encoding human- and machine-readable files)