Changing patterns in reporting and sharing of review data in systematic reviews with meta-analysis: the REPRISE project

Objectives: To examine changes in completeness of reporting and frequency of sharing review materials in systematic reviews (SRs) over time; and factors associated with these changes. Methods: We examined a random sample of 300 SRs with meta-analysis indexed in PubMed, Science Citation Index, Social Sciences Citation Index, Scopus and Education Collection in November 2020. We compared the extent of complete reporting in these reviews against 110 SRs indexed in MEDLINE in February 2014. We examined associations between completeness of reporting and various factors (e.g. self-reported use of reporting guidelines, journals data sharing policies) by calculating risk ratios (RR) and 95% confidence intervals (CI). Results: Among 300 SRs from 2020, authors of only 7% shared their review data file(s) and 1% shared analytic code. Compared to 2014 sample, reviews in 2020 were more likely to reference reporting guidelines in their manuscript (RR=2.8, 95% CI 2.1-3.8), report the full search strategy for at least one database (RR=1.3, 95% CI 1.1-1.6) and methods of data preparation for meta-analysis (RR=2.2, 95% CI 1.4-3.5). Among reviews in 2020, those for which authors mentioned using reporting guidelines reported review protocols, study screening processes and numbers of records retrieved for each database more frequently than those that did not mention reporting guidelines; however the 95% CIs for these associations included the null. Reviews published in journals that mandated either data sharing or inclusion of Data Availability Statements were more likely to share their review materials (e.g. data, code files) (RR=8.1, 95% CI 3.1-21.5). Conclusion: There was a notable increase over time in self-reported use of a reporting guideline, but we were uncertain whether it was associated with improved reporting of SRs. Data sharing policies of journals potentially encourage sharing of review materials. Further studies are needed to explore other facilitators or barriers to complete reporting in SRs.


INTRODUCTION
54 Systematic reviews provide a design for identifying, critically appraising and synthesising all available evidence 55 addressing a specific research question. A systematic review may also include meta-analyses, which is the 56 statistical synthesis of results across studies to produce a reliable estimate of the true effect. The recognition 57 of systematic reviews' role in evidence-based clinical practice and policymaking is reflected in the 20-fold 58 increase in the number of published systematic reviews between 2000-2019 (1).

59
To ensure systematic reviews are valuable to decision makers, authors should completely report the methods 60 and results of their review. Complete reporting allows users to judge whether the chosen methods may have 61 biased the review findings (e.g. if inclusion of studies is restricted to only those written in English). Incomplete 62 reporting of the methods prevents such an assessment and can preclude attempts to replicate the findings.

63
Several meta-research studies have evaluated the completeness of methods and results reporting in 64 systematic reviews and meta-analyses. Many of these have been narrow in scope, focusing on specific 65 medical specialties, health topics (2-6) or selected journals (7,8). While other studies have examined a more 66 diverse sample of reviews (9,10), the cohorts of reviews examined are no longer current.

67
To address incomplete reporting of methods and results in systematic reviews, several reporting guidelines 68 have been developed. Reporting guidelines provide a structure for reporting a systematic review, along with 69 recommendations of items to report (11). Key reporting guidelines for systematic reviews include the 70 Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement (12)

73
Evidence from studies examining adherence to PRISMA suggest that for some items (e.g. inclusion of a flow 74 diagram) there was improvement after the introduction of the PRISMA statement in 2009, but that others (e.g. 75 mention of a review protocol) remained infrequently reported (13). Similarly, improvements in the reporting of 76 network meta-analysis (NMA) were observed after the introduction of the PRISMA-NMA extension (16); 77 however, the authors concluded that the improvements could not be attributed solely to the availability of this 78 extension. The extent to which adoption of reporting guidelines influences completeness of reporting remains 79 unclear.

80
In addition to complete reporting of methods and results, advocates for greater transparency of systematic 81 reviews (17,18) also recommend authors share systematic review data, analytic code used to generate meta-. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 18, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022 Page 4 of 22 82 analyses, and other materials used in the review (e.g. lists of citations screened, data collection templates).

83
Access to the data files facilitates independent verification of findings and updating of reviews when new 84 evidence becomes available (19). Infrequent sharing of data in systematic reviews in health research has been 85 observed, but these findings may not generalise to all health topics (4) or across journals (10). Moreover, the 86 types of data shared (e.g. unprocessed data extracted from reports, data included in meta-analyses) has not 87 been examined.

88
Many journals and funders now require, or encourage, study data to be shared, and in journal article 89 submissions, a statement provided regarding the availability of data (known as a Data Availability Statement) 90 (20). Some studies have explored the impact of such data policies (often termed 'open data policies') on the 91 frequency of data availability statements and data files among academic papers (21-24). However, none of 92 these studies included systematic reviews, and therefore their findings may not be generalisable to reviews, 93 which produce different types of data to that generated from clinical trials and laboratory experiments.

94
In view of the research gaps outlined above, we aimed to: 95 (a) evaluate the completeness of reporting in systematic reviews in a cross-section of systematic reviews with 96 meta-analysis published in 2020; 97 (b) evaluate the frequency of sharing review data, analytic code and other materials in the same cohort of 98 reviews; 99 (c) compare reporting in these reviews with a sample of reviews published in 2014; 100 (d) investigate the impact of reporting guidelines on the completeness of reporting; and 101 (e) investigate the impact of data sharing policies of journals on the frequency of review data sharing.

104
This study was conducted as one of a suite of studies in the REPRISE (REProducibility and Replicability In 105 Syntheses of Evidence) project. The REPRISE project is investigating various aspects relating to the 106 transparency, reproducibility and replicability of systematic reviews with meta-analysis of the effects of health, 107 social, behavioural and educational interventions (25). Methods for all studies were pre-specified in the same 108 protocol (25).

109
Identification and selection of articles . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted April 18, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022 We included a random sample of systematic reviews with meta-analysis of the effects of a health, social, 111 behavioural or educational intervention in our sample (see S1 Appendix for eligibility criteria). To be considered 112 a "systematic review", authors needed to have, at a minimum, clearly stated their review objective(s) or 113 question(s); reported the source(s) (e.g. bibliographic databases) used to identify studies meeting the eligibility 114 criteria; and reported conducting an assessment of the validity of the findings of the included studies, for 115 example via an assessment of risk of bias or methodological quality. We did not exclude systematic reviews 116 providing limited detail about the methods used. We only included systematic reviews that presented results 117 for at least one pairwise meta-analysis of aggregate data. Systematic reviews with network meta-analyses 118 were eligible if they included at least one direct (i.e. pairwise) comparison that fulfilled the above-mentioned 119 criteria. Systematic reviews with only meta-analyses of individual participant data were excluded. Furthermore, 120 only reviews written in English were included.

127
We used Endnote v9.3.3 for automatic deduplication of records, then randomly sorted unique records in 128 Microsoft Excel using the RAND() function, and imported the first 2,000 records yielded from the search into 129 Covidence (26) for screening. Two authors (MJP and either PN or RK) independently screened the titles and 130 abstracts of the 2,000 records against the eligibility criteria. We retrieved the full text of all records deemed 131 potentially eligible, and two authors (PN and either MJP or RK) independently evaluated them in random order 132 against the eligibility criteria until we reached our target sample size of 300 systematic reviews. Any 133 disagreement at each stage of screening was resolved via discussion or adjudication by the senior reviewer 134 (MJP). A sample of 300 systematic reviews allowed us to estimate the percentage of reviews reporting a 135 particular practice within a maximum Wald margin of error of 6%, assuming a prevalence of 50%; for a 136 prevalence of less (or greater) than 50%, the margin of error will be smaller.

139
Two authors (PN and either MJP, RK or ZA) collected data independently and in duplicate from all of the 300 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

141
Any disagreement in the data collected was resolved via discussion or adjudication by the senior reviewer 142 (MJP). Prior to data collection, a pilot test of the data collection form was performed on a random sample of 143 10 systematic reviews in order to discuss any discrepancies and adjust the form as necessary. The full data 144 collection form (S3 Appendix) includes a subset of items captured by Page et al. in their previous evaluations 145 of completeness of reporting (9,10), along with additional items to capture some issues not previously 146 examined. The wording of items in the data collection form was matched to that used in previous evaluations 147 (9,10) to facilitate comparison.

148
The form consisted of three sections. The first section captured general characteristics of the review (e.g.

153
The second section consisted of items characterising completeness of reporting and types of review materials 154 being shared. For example, we recorded whether a full search strategy (i.e. including Boolean logic operators 155 such as 'AND', 'OR', 'NOT' etc.) was described for each electronic database searched;; whether summary 156 statistics and effect estimates for each study included in the index meta-analysis (defined as the first meta-157 analysis whose results were mentioned in the Abstract, or if not applicable, the Results section) were reported; 158 and whether data files and analytic code underlying the meta-analyses reported were publicly available and 159 how to access them. To facilitate our analysis of the impact of reporting guidelines on completeness of 160 reporting, we also recorded whether the authors self-reported using a reporting guideline, defined as any 161 document specifying essential items to report in a systematic review (e.g. PRISMA, MECIR or MECCIR 162 standards, etc.)

163
The final section captured the data sharing policy of the journal where the article was published. A data sharing 164 policy refers to the journal's requirements and expectations regarding public sharing of data and code used in 165 the review. We examined each journal's instructions to authors and submission guidelines to capture whether 166 the journal has a mandatory requirement for authors to include a Data Availability Statement, or to share all 167 data and code used in the review, or both (30). These requirements correspond to level I-II of the Transparency 168 and Openness Promotion Guidelines for data transparency (31). We also recorded whether the study was 169 published in an evidence synthesis journal, defined as a journal that published exclusively systematic reviews 170 and their protocols (e.g. Cochrane Database of Systematic Reviews, The Campbell Library, JBI Evidence . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 18, 2022. ; https://doi.org/10.1101/2022.04.11.22273688 doi: medRxiv preprint Synthesis or Obesity Reviews). All journal policy information was sought in August 2021.

172
We collected data from the main report of the systematic review, any supplementary file provided on the journal 173 server or any cited repository, as well as the review protocol if the authors specified that the relevant 174 information was contained therein. In the event of discrepancies between the protocol and the main report, we 175 gave preference to data from the main report.  (36)). We 198 define the equivalence range for all comparisons as 0.9 to 1.1 -any RR less than 0.9 or more than 1.1 is 199 deemed as an important difference.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 18, 2022. ; https://doi.org/10. 1101/2022 Page 8 of 22

200
We conducted two post-hoc sensitivity analyses, the first by excluding Cochrane reviews because they were 201 subjected to strict editorial processes to ensure adherence to methodological conduct and reporting standards,

221
The included reviews covered a wide range of topics. In nearly all reviews (n=294, 98%), the intervention was 222 classified as a health intervention, and in 37 (12%) as a behavioural, social or educational intervention in 37 223 (12%) reviews (some reviews examined both types of interventions). Almost two-thirds of the reviews (n=198, 224 66%) examined the effects of non-pharmacological interventions. Out of 24 ICD-11 categories of diseases and 225 conditions, our sample of reviews captured 23 categories. The top four categories (endocrine, nutritional or 226 metabolic diseases, diseases of the digestive system, the musculoskeletal system, and the circulatory system) 227 accounted for 46% of all systematic reviews.

228
The included systematic reviews were published across 224 journals. Five journals (accounting for 5% of all 229 systematic reviews) specialised in evidence synthesis; 141 journals (accounting for 66% of all systematic . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 18, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022 Page 9 of 22 230 reviews) outline a data-sharing policy in the instruction page for authors (S5 Appendix).

231
The general characteristics of the 2014 sample have been described elsewhere (10). In brief, the 2014 sample 232 consisted of 110 systematic reviews indexed in MEDLINE in February 2014, and was similar to the 2020 233 sample in many aspects, such as the sample size of each review (median=13 studies, IQR 7-23), size of the 234 index meta-analysis (median=6 studies, IQR 3-11) and the prevalence of non-pharmacological reviews (n=55, 235 50%). Similar to the 2020 sample, the reviews in 2014 were published in a wide range of journals (n=63), 236 addressed several clinical topics (19 ICD-10 categories) and were predominantly led by authors from China, 237 the UK and Canada (n=55 combined, 50%).

244
A large number of items were reported in less than 50% of reviews. These include a registration record (38%) 245 or protocol (4%) for the review, the interfaces used to search databases (e.g. Ovid, EBSCOhost) (37%), exact 246 start and end dates of the search range (45%), search strategy for sources that are not bibliographic databases 247 (17%), number of records retrieved for each database (42%), citation for at least one excluded article (22%), 248 methods of data preparation (data conversion, calculation of missing statistics, etc.) (34%) and the 249 heterogeneity variance estimator used for meta-analysis (21%).

250
In terms of sharing review materials, 20 systematic reviews (7%) made data files or analytic code underlying 251 the meta-analysis publicly available, which included two reviews (1%) that shared analytic code. All of these 252 reviews shared these data via supplementary files; two reviews additionally hosted data and analytic code in

257
We compared 294 systematic reviews of health interventions from our present 2020 sample with 110 reviews 258 in the 2014 sample. Compared to the 2014 reviews, systematic reviews indexed in 2020 more frequently cited 259 a reporting guideline to guide their reporting (RR=2.8, 95% CI 2.1 to 3.8). In addition, systematic reviews in 260 2020 were more likely to report a full search strategy for at least one database (RR=1.3, 95% CI 1.1 to 1.6), . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

266
Given the notable discrepancy in the number of Cochrane reviews between the 2014 and 2020 samples, we 267 conducted a sensitivity analysis in which we excluded Cochrane reviews from both samples (8 reviews from 268 2020 sample and 32 reviews from 2014 sample). Three main changes were observed compared to the original 269 analysis. Firstly, existing differences became more pronounced, as observed for citing a reporting guideline 270 (RR=2.0, 95% CI 1.5 to 2.6), full search strategy for at least one database (RR=2.0, 95% CI 1.5 to 2.7) and 271 data preparation method(s) (RR=2.1, 95% CI 1.2 to 3.6). Secondly, uncertainty was eliminated for some items, 272 whose 95% CI now exceeded the equivalence range: non-Cochrane reviews in 2020 were more likely to report 273 a protocol registration record (RR=4.5, 95% CI 2.2 to 9.2), funding source (RR=1.3, 95% CI 1.1 to 1.6) and 274 dates of coverage for the databases (RR=1.3, 95% CI 1.1 to 1.6). Lastly, after excluding Cochrane reviews, 275 the rate of sharing review data and code of reviews in 2020 became higher than those of 2014, although the

282
Of the 300 reviews, 245 (82%) reported using a reporting guideline. Systematic reviews using a reporting 283 guideline appeared to report more frequently a published or registered protocol (RR=1.3, 95% CI 0.9 to 2.0), 284 methods used in study screening (RR=1.2, 95% CI 1.0 to 1.4) and number of records retrieved for each 285 database (RR=1.4, 95% CI 0.9 to 2.2). The confidence intervals included and extended beyond the 286 equivalence range, suggesting that there could be important differences.

287
. For other items, however, evidence that reporting guidelines improved reporting was inconclusive, either 288 because the confidence interval extended beyond the equivalence range in both directions, or the upper limits 289 did not exceed 1.1 We conducted a sensitivity analysis by excluding systematic reviews on  290 from both groups, but no notable changes were observed (Fig. 3A-B).

291
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 18, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022  Impact of journals on patterns of reporting in systematic reviews from 2020 296 Only 14 systematic reviews from 2020 were published in specialist evidence synthesis journals, including eight 297 Cochrane reviews. Compared to 286 systematic reviews from 2020 published in non-specialist evidence 298 synthesis journals, these reviews were more likely to report a protocol (RR=1.8, 95% CI 1.2 to 2.5), a full 299 search strategy for at least one database (RR=1.3, 95% CI 1.1 to 1. 6), details of the screening process 300 (RR=1.3, 95% CI 1.2 to 1.4) and ROB assessment processes (RR=1.4, 95% CI 1.1 to 1.8), ROB results for 301 individual studies (RR=1.3, 95% CI 1.2 to 1.4) and data preparation methods (RR=2.2, 95% CI 1.5 to 3.3) (Fig.   302   4). In addition, these systematic reviews performed better in reporting items related to the search methods 303 such as the interface used to search bibliographic databases (RR=2.2, 95% CI 1.6 to 3.0), search strategy for 304 non-database sources (trial registers, Google Scholar, etc.) (RR=6.0, 95% CI 3.4 to 10.5), date of last search 305 (RR=3.7, 95% CI 2.6 to 5.2) as well as extra details of search results such as citation for at least one excluded 306 study (RR=3.3, 95% CI 2.1 to 5.2) and the number of retrieved records from each database (RR=1.8, 95% CI 307 1.2 to 2.5). They were also more likely to share relevant data and materials (RR=28.7, 95% CI 8.6 to 95.3). 308 309 Fig. 4. Relationship between journal type and reported items 310 311 A journal's mandatory requirement for data sharing or declaration of data availability was associated with a 312 larger percentage of data or materials being shared with the published systematic review (RR=8.1, 95% CI 3.1 313 to 21.5). A larger percentage was also observed for these specific types of material: unprocessed data 314 collected from included studies (RR=3.9, 95% CI 1.1 to 14.1) and data files used in analysis (RR=14.6, 95% 315 CI 3. 6 to 60.0). .Results for other types of materials, such as files showing data conversions, analytic code 316 and citations of all screened studies, are inconclusive due to small sample size (Fig. 5). Similar findings were 317 observed when comparing between journals with any data-sharing policy (mandatory or not) and journals 318 without one (S1 Fig.).

DISCUSSION
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 18, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022 Page 12 of 22 323 Findings from our examination of 300 randomly selected systematic reviews indexed in 2020 indicate 324 infrequent reporting of certain items, particularly the availability of a review protocol or registration entry, search 325 strategy for all databases searched, methods of handling data (e.g. data conversions), and sharing of meta-326 analytic data. This aligns with previous evaluations (2,4,7,38). The scarcity of data and code availability is also 327 echoed by other studies (6,8).

329
On reporting of systematic reviews 330 There are two possible reasons why we observed few notable improvements in reporting between 2014 and 331 2020. Many items were already reported frequently in 2014 (e.g. reporting of competing interests, eligibility 332 criteria, meta-analytic models, effect estimate for each study), leaving little opportunity for improvement. This 333 is reflected by the observation that the greatest improvements were found in items that had been reported by 334 less than 50% of the systematic reviews in 2014 (Fig. 2). Furthermore, Cochrane reviews made up a smaller 335 proportion in our 2020 sample (n=8, 3%) compared to 2014 (n=32, 29%), suggesting a rise in popularity of 336 systematic reviews being produced outside of the auspices of an organisation with editorial processes 337 dedicated to ensuring complete reporting. Our sensitivity analysis, which excluded Cochrane reviews, yielded 338 larger estimates of differences between 2014 and 2020 reviews, suggesting that the improvements between 339 2014 and 2020 are driven by changes in non-Cochrane reviews. This is highly possible given that Cochrane 340 reviews, owing to their stringent reporting requirements, tend to have less room for improvement than non-341 Cochrane reviews.

342
More systematic reviews in 2020 than 2014 cited a reporting guideline yet citing a reporting guideline was not 343 clearly associated with more complete reporting for most items evaluated. This challenges the assumption that 344 referencing a reporting guideline means that the authors have consulted the guideline while reporting the 345 review. In reality, however, several factors could have affected the authors' decision to report (or not to report) 346 certain items. Firstly, authors might not recognise that some items consist of multiple elements that need to be 347 reported in details. For example, we observed among our sample a tendency to report the reviewer 348 arrangement only for the study selection stage, not the subsequent data collection or ROB assessment stages. . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 18, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022 In view of these findings, we recommend interviews be conducted with review authors to explore their 355 understanding of reporting guidelines and identify challenges in reporting of reviews. There are certainly needs 356 for development of tools that facilitate the reporting process. For example, a computer-based tool to break 357 down the PRISMA reporting recommendations into digestible steps would be useful for first-time reviewers.

358
Another example would be a guide specifying which details of the meta-analytic model to report on a forest 359 plot, and how to accomplish that in various meta-analytic softwares. Lastly, this raises the discussion on 360 whether a universal mechanism should be applied during peer review to verify that authors adhered to all items 361 of the reporting guideline they cited, and how to effectively enforce this.

363
On data sharing in systematic reviews

364
Our study demonstrates that presence of a data sharing policy by the publishing journal had positive impacts 365 on the frequency of sharing certain types of data necessary for replicating systematic reviews. The overall rate 366 of data and code sharing, however, leaves much to be desired. While two-third of our reviews (66%) were 367 subjected to some form of data policy by the journal in which their review was published, less than a third 368 (32%) complied to the policy by including a Data Availability Statement or actually sharing any data files; only 369 7% shared at least one type of file. A meta-research study featuring a similar percentage of journals with open 370 data policies (70%) found similar results (9% shared data or code, and 29% either shared data or included a 371 Data Availability Statement (or both)) (39).

372
The low rate of data and code sharing can be attributed to several factors. Firstly, the larger number of non-373 Cochrane reviews in 2020 explains the less frequent availability of data and materials used for analysis in this 374 later sample (compared to 2014), as authors of Cochrane reviews routinely share RevMan files containing 375 meta-analysis data, which were rarely made available for other reviews. Secondly, there are motivational and 376 technical barriers to data sharing that cannot be sufficiently addressed by data sharing policies (40). For 377 example, while journals can encourage data sharing and suggest available options for data upload, reviewers 378 may lack the technical expertise and time to prepare and annotate their files for public view. Moreover, 379 concerns about data ownership, lack of incentives and fear of criticism can impede motivations to share review 380 data. Some studies have explored these barriers in general academia (40,41), but we are uncertain whether 381 researchers in evidence synthesis will face all of these barriers or even unidentified barriers unique to 382 systematic reviews and meta-analyses. Future studies in the REPRISE project will explore systematic 383 reviewers' perspectives on barriers and incentives to reporting and data-sharing in order to address these 384 questions (25).
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review)
The copyright holder for this preprint this version posted April 18, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022 Lastly, our findings also highlight the important role of supplementary files or public repositories for data 386 sharing in systematic reviews. Supplementary files and public repositories enable authors to share data and 387 materials necessary to validate the review process while keeping the main article concise and relevant to lay 388 readers. For example, authors can outline in a separate file the database-specific search strategy, number of 389 records retrieved and date of last search for each database consulted. We should endeavour to change the 390 notion that this information is less important and make sharing of review data and materials via 391 supplementary files or public repositories a standard practice for systematic reviews. In order to achieve this, 392 concerted efforts are needed to standardise data structures, establish fair use guidelines and create a 393 supportive environment for collaboration in the systematic review community.

396
Although several other meta-research studies have explored completeness of reporting and the frequency of 397 data sharing in systematic reviews (2)(3)(4)(5)7,8), our study offers several methodological advantages. Our sample 398 was obtained from several databases, and was not limited to a specific topic or journal. Our study captured 399 not only the frequency of data sharing, but also the type of systematic review data, code and materials being 400 shared. All stages of study screening and data collection were conducted by two authors independently, which 401 minimised random and systematic errors and increased accuracy. Lastly, we directly compared our 2020 402 sample with a 2014 sample that was retrieved and evaluated using the same criteria (9,10), thus minimising 403 the impact of methodological variations on temporal trends.

404
Nonetheless, our study was not without limitations. As a cross-sectional study, our results should be viewed 405 as generating hypotheses rather than proving a causal association (e.g. between data sharing policies and 406 actual sharing of systematic review data). Some items were reported by fewer than 50 reviews, such as sharing 407 of data & materials, heterogeneity variance estimator used, and a search strategy for non-bibliographic 408 sources. This leads to uncertainty in interpreting their risk ratios, as the 95% confidence intervals were either 409 too wide or crossed the null value (or both). Future studies should be conducted specifically for systematic 410 reviews reporting these items to ensure adequate power. Despite intending to include systematic reviews of 411 the effects of health, social, behavioural and educational interventions, the vast majority of reviews evaluated 412 the effects of a health intervention. Therefore, our findings are less generalizable to systematic reviews of the 413 other types of interventions. Lastly, our findings do not necessarily generalise to systematic reviews indexed 414 in databases other than the ones we searched or to systematic reviews written in languages other than English.
. CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted April 18, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022