ABSTRACT
Objectives To examine changes in completeness of reporting and frequency of sharing data, analytic code and other review materials in systematic reviews (SRs) over time; and factors associated with these changes.
Design Cross-sectional meta-research study
Sample A random sample of 300 SRs with meta-analysis of aggregate data on the effects of a health, social, behavioural or educational intervention, which were indexed in PubMed, Science Citation Index, Social Sciences Citation Index, Scopus and Education Collection in November 2020.
Analysis/Outcomes The extent of complete reporting and frequency of sharing review materials in these reviews were compared with 110 SRs indexed in February 2014. Associations between completeness of reporting and various factors (e.g. self-reported use of reporting guidelines, journal’s data sharing policies) were examined by calculating risk ratios (RR) and 95% confidence intervals (CI).
Results Among 300 SRs from 2020, authors of only 7% shared their review data file(s) and 1% shared analytic code. Compared to 2014 sample, reviews in 2020 were more likely to reference reporting guidelines in their manuscript (RR=2.8 [2.1 to 3.8]), report the full search strategy for at least one database (RR=1.3 [1.1 to 1.6]) and methods of data preparation for meta-analysis (RR=2.2 [1.4 to 3.5]). Among reviews in 2020, those for which authors mentioned using reporting guidelines reported review protocols, study screening processes and numbers of records retrieved for each database more frequently than those that did not mention reporting guidelines; however the 95% CIs for these associations included the null. Reviews published in journals that mandated either data sharing or inclusion of Data Availability Statements were more likely to share their review materials (e.g. data, code files) (RR=8.1 [3.1 to 21.4]).
Conclusion Reporting in systematic reviews has improved between 2014-2020 for some reporting items. Frequency of sharing review materials remains low. There was an increase in self-reported use of a reporting guideline; however, it is uncertain whether citing a reporting guideline was associated with improved reporting. Data sharing policies of journals potentially encourage sharing of review materials.
SUMMARY BOX
What is already known on this topic
What is already known on this topic
Complete reporting of methods and results, as well as sharing data and analytic code, enhances transparency and reproducibility of systematic reviews. The extent of complete reporting and sharing data or analytic code among systematic reviews needs to be comprehensively assessed.
Use of reporting guidelines, which are designed to improve reporting in systematic reviews, is increasing. It is unclear whether this increase had an impact on reporting of methods and results in systematic reviews.
More journals are adopting open data policies which aim to promote data sharing. The impact of these policies on sharing data and analytic code in systematic reviews is also unclear.
What this study adds
What this study adds
Reporting in systematic reviews has improved between 2014-2020, especially for items that had previously been infrequ ently reported. Frequency of sharing review data and analytic code remains low.
Our study found an increase in self-reported use of a reporting guideline between 2014-2020; however, citing a reporting guideline was only associated with improved reporting of certain items.
Our findings suggest data sharing policies of journals had a positive impact on sharing of review data and analytic code.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This research is funded by the Australian Research Council Discovery Early Career Researcher Award (DE200101618), held by MJP; JEM is supported by the National Health and Medical Research Council Career Development Fellowship (APP1143429); DM is supported in part as the University Research Chair, University of Ottawa; NRH is funded by the Alexander von Humboldt Experienced Researcher Fellowship; DGH is supported by the Australian Commonwealth Government Research Training Program Scholarship; RK is supported by the Monash Graduate Scholarship and the Monash International Tuition Scholarship. The funders had no role in the study design, decision to publish, or preparation of the manuscript.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
We have clarified several of the methods, shortened the Introduction and Discussion sections, moved one of the supplementary tables into the manuscript, and specified deviations from the protocol
Data Availability
All datasets and analytic code can be found on the Open Science Framework (DOI: 10.17605/OSF.IO/JSP9T).
