Methods used to select results to include in meta-analyses of nutrition research: a meta-research study

Objectives: To investigate the extent of multiplicity of results in study reports of nutrition research, and the methods specified in systematic reviews to select results for inclusion in meta-analyses. Methods: MEDLINE and Epistemonikos were searched (January 2018 - June 2019) to identify systematic reviews with meta-analysis of the association between food/diet and health-related outcomes. A random sample of these reviews was selected, and for the first presented ('index') meta-analysis, rules used to select effect estimates to include in this meta-analysis were extracted from the reviews and their protocols. All effect estimates from the primary studies that were eligible for inclusion in the index meta-analyses were extracted. Results: Forty-two systematic reviews were included, 14 of which had a protocol. In 29% of review protocols and 69% of reviews, at least one decision rule to select effect estimates when multiple were available was specified. In 69% (204/325) of studies included in the index meta-analyses, there was at least one type of multiplicity of results. Conclusions: Authors of systematic reviews of nutrition research should anticipate encountering multiplicity of results in the included primary studies. Specification of methods to handle multiplicity when designing reviews is therefore recommended.


Background:
The Global Burden of Disease study 2019 reported that diet has a significant impact on health outcomes. Diet quality was found to be the fifth leading risk factor for disability adjusted life years [1]. Large and long-term prospective observational studies and short-term clinical trials have found associations between a particular dietary factor and non-communicable diseases [2][3][4]. Systematic reviews (SRs) based on such studies are being used to inform recommendations in dietary guidelines [5][6][7]. However, flaws in the design, conduct and reporting of SRs may yield misleading results, and in turn, misinform guideline recommendations [8].
One challenge SR authors often face is a multiplicity of results in the primary studies [9]. For example, a study report may present multiple effect estimates for the association between red meat consumption and gout, where these estimates may arise from the fitting of multiple statistical models with different outcome definitions of gout, different exposure levels, or where adjustment is made for different sets of confounders. While inclusion of multiple effect estimates from a particular study in a meta-analysis is possible (using methods that adjust for statistical dependency) [10], more commonly only one of the available effect estimates is selected for inclusion. There are various methods that can be used to select a single effect estimate [9,10]. However, when this selection is based on the statistical significance, magnitude or direction of effect, this may introduce bias into the meta-analysis effect estimate [11]. We refer to this selection process as 'selective inclusion of results'.
To help mitigate the potential for selective inclusion of results, it has been recommended that methods for dealing with multiplicity should be pre-specificied [11]. This includes prespecification of "eligibility criteria" for each meta-analysis, indicating which results are . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint eligible for inclusion in the meta-analysis (e.g. intervention groups, measurement instruments, time points), and "decision rules", which specify the methods that will be used to select a single result from a study when multiple are eligible for inclusion in the same meta-analysis (see Box 1 for examples of eligibility criteria and decision rules).

Box 1. Examples of eligibility criteria and decision rules to select results
Example of eligibility criteria to select results: Systematic reviewers state that only study effect estimates that were adjusted for sex and age would be included in a meta-analysis of the association between fruit consumption and coronary heart disease.
Example of a decision rule to select results when multiple are available: Systematic reviewers state that if multiple effect estimates quantifying the association between different levels of fruit consumption and coronary heat disease were available, as would arise if intake was categorised into quartiles in a study report, only the contrast between the highest (e.g. quartile 4) and lowest (e.g. quartile 1) intake would be included in the metaanalysis.
Previous research has examined the extent of multiplicity of results, and the methods used to select results for inclusion in the meta-analysis [11][12][13]. These studies have focused on a range of conditions and examined multiplicity in meta-analyses of randomised trials with continuous outcomes. All studies found that multiplicity of results was common, and Page et al. [9] and Tendal et al. [12] found that specification of methods to select results for inclusion were rarely reported. However, none of the studies examined meta-analyses in nutiriton research, inclusion of randomized or non-randomized studies, or where the outcome was noncontinuous (e.g. binary, count or time-to-event); circumstances for which different forms of multiplicity might arise. Therefore, we aimed to address this gap and investigate the i) extent . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint of multiplicity of results in sudy reports of nutrition research, and ii) the methods specified in systematic reviews to select results for inclusion in meta-analyses of all outcome types, including randomized or non-randomized study designs.

Methods
The study protocol has been published [14]. Here, we provide an overview of the methods, with modifications to the protocol reported in Supplementary Table 1.

Eligibility criteria, search and selection of SRs
SRs that satisfied the definition of an SR, as outlined in the 2019 edition of the Cochrane Handbook for Systematic Reviews of Interventions [15], and that had explicitly stated methods of study identification (e.g. a search strategy) and of study selection (e.g. eligibility criteria and selection process), and included a meta-analysis of study results, were eligible for inclusion in this study. We included such SRs with meta-analysis that: • included studies that enrolled, regardless of their age and background, (a) people who were generally healthy, (b) a mixture of generally healthy people and people with dietrelated risk factors (e.g. overweight, high blood pressure) or a particular health condition (e.g. type II diabetes or cardiovascular disease), or (c) people with nonspecified health status; • included randomized trials or non-randomized studies that assessed the effects of at least one type of food (e.g. eggs, fish) or at least one food-defined dietary pattern (e.g. Mediterranean diet) on any continuous (e.g. systolic blood pressure) or non-continuous (e.g. gout incidence) health-related outcome; • were published between 1 January 2018 and 30 June 2019; • were written in English; . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint • provided citations for all included studies in the SR, and; • presented the summary statistics or effect estimate and its precision (e.g. standard error or 95% confidence interval) for each included study, and the meta-analytic summary effect estimate and its precision in the text or forest plot, for at least one meta-analysis of a continuous or non-continuous outcome.
We excluded: • SRs that did not include any meta-analysis of a non-continuous or continuous outcome; • meta-analyses or pooled analyses of studies conducted outside the context of a SR; • SRs that only focused on nutrient-specific associations with outcomes (e.g. examining the effects of single nutrients such as folic acid, salt); • SRs that included studies enrolling only participants with a health condition, who were obese or who were frail or elderly people at risk of malnutrition, and; • SRs that were co-authored by any of our research team members. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint analyses of continuous and 25 meta-analyses of non-continuous outcomes. If the total number of eligible SRs exceeded this target at the end of a batch, we planned to randomly sample 25 SRs of each type. Any discrepancies in screening decisions at each stage were resolved via discussion between investigators, or by consultation with another investigator (JM) where necessary. For each included SR, we retrieved the published protocol for the SR or registration record (e.g. PROSPERO record), if available, as cited or reported in the SR.
From each SR meeting the inclusion criteria, one investigator (MJP) selected one metaanalysis for inclusion. The selected meta-analysis was the first meta-analytic result mentioned in the review (with no restrictions on its placement in the manuscipt); we refer to the selected meta-analysis as the "index meta-analysis". We initially selected an index meta-analysis regardless of the outcome domain (e.g. weight, bladder cancer), effect measure (e.g. odds ratio, standardised mean difference), meta-analytic model and number and type of included studies (i.e. randomized or non-randomized study). However, following selection of 50 index meta-analyses, we decided to restrict our inclusion to only those meta-analyses including fewer than 20 studies, for reasons of feasibility. For each included index meta-analysis, we retrieved the reports of all included studies (see the protocol for further details [14]).

Data collection and management
A data collection form was developed in REDCap (see Supplementary Table S2) [17]. Seven is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint on the remaining index meta-analyses and their included studies. Any discrepancies were resolved through discussions between two investigators or adjudication by a third investigator (JM) if necessary.
An overview of the data items, and the sources these were obtained from (i.e. systematic review protocol, systematic review or study report), is presented in Table 1; further details are available in the protocol [14]. In the case of data extracted from the reports of studies included in the index meta-analysis, we extracted all outcome data that were eligible for inclusion in the index meta-analysis. This was determined by the eligibility criteria and decision rules stated in the SR protocol if available, and if not available, the SR, and in combination with the comparison and outcome of the meta-analysis. For example, if the systematic reviewers pre-defined in the SR protocol that the eligible intervention and comparator for the meta-analysis of weight gain was "highest versus lowest intake of dairy products", and pre-defined a decision rule stipulating that they would consider only data at 12 weeks follow-up when data were available at multiple time points, we only extracted data for that comparison and time point, regardless of whether study reports had data for other time points and other comparisons for the same outcome. In the absence of an SR protocol, we assumed that no eligibility criteria and decision rules were pre-specified ('worst-case scenario' assumption) and extracted all study outcome data based on how the outcome was specified in the SR. For example, if the systematic reviewers did not state in a protocol which results should be selected when multiple were available, yet defined the meta-analysis as "effect of dairy intake on weight at 6 months", we extracted all data on weight at 6 months, regardless of the level of intake of dairy, whether results were unadjusted or adjusted, or what analysis sample was used. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint

Data analysis
We used descriptive statistics to summarise the characteristics of SR protocols, SRs, index meta-analyses and included studies. For categorical variables, we present frequencies and percentages. For continuous variables, we report medians (with interquartile ranges [IQR]).
We computed the frequencies and percentages for different types of eligibility criteria and decision rules used to select results, differences in eligibility criteria or decision rules between the SR protocol and the SR, and studies with different types of multiplicity of results. We also calculated risk differences (with 95% confidence intervals [CIs]) to examine whether the percentages of different types of eligibility criteria and decision rules and studies with different types of multiplicity of results, differed between meta-analyses of continuous outcomes and of non-continuous outcomes. Risk differences were calculated using the method of Mee with the Miettinen and Nurminen modification. Analyses were undertaken using the statistical packages Stata (College Station, Tx), except for the calculaton of risk differences, which was undertaken using the library PropCIs [18] in R (Vienna, Austria) [19].

Results of search and screening
Our search yielded a total of 7,167 references from the PubMed and Epistemonikos databases is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint non-continuous), because the index meta-analysis had 20 or more studies, leaving 42 included SRs .

Characteristics of the included systematic reviews and their index meta-analyses
Of the 42 SRs, 14 had accessible protocols (2 were published protocols and 12 were PROSPERO records) ( Table 2). In most index meta-analyses the population was unclear (45%, 19/42), 33% (14/42) included only healthy participants and the remainder (22%, 9/42) included a mix of healthy people and people with a health condition. The majority of index meta-analyses included only non-randomized studiess (62%, 26/42), 31% (13/42) included only randomized trials and the remaining 7% included studies of both designs ( Table 3). The primacy of the outcome was not identified in most reviews (81%, 34/42). In nearly all metaanalyses, a random-effects model was fitted (90%, 38/42). The 42 index meta-analyses included a total of 325 studies, with a median of seven studies (IQR 5-11; range 2-17) per meta-analysis.

Eligibility criteria and decision rules reported in SR protocols
Of the SR protocols (n=14), all included at least one eligibility criterion, and four (29%) reported at least one decision rule to select results (  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

Eligibility criteria and decision rules reported in SRs
The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint Of the SRs (n = 42), all included at least one eligibility criterion and 69% reported at least one decision rule to select results (Table 5). Similar to the SR protocols, the most commonly reported eligibility criteria (95%, 40/42) and decision rule (40%, 17/42) in the SRs was based on interventions/exposures. Eligibility criteria and decision rules for the type of analysis were more freqently specified in SRs as compared with their protocols. The most commonly reported decision rules for analyses were rules to select from multiple unadjusted and covariate-adjusted analyses (24%) ( Table 5). There were some discrepancies observed between SR protocols and their published SRs, with the most common type being the addition of a new decision rule to deal with multiple unadjusted and covariate-adjusted analyses in the included studies (Supplementary Table S4).
The percentage of reviews specifying different types of eligibility criteria and decision rules generally did not differ by outcome type. However, a larger percentage of SRs with an index meta-analysis of a continuous outcome specified eligibility criteria for any type of analysis compared with SRs with a non-continuous outcome (37% vs 17%; risk difference [RD] 20%, 95% CI -7% to 45%). Conversley, a smaller percentage of SRs with an index meta-analysis of a continuous outcome specified a rule for selecting an adjusted/unadjusted result compared with SRs of a non-continuous outcome (11% vs 35%; RD -24%, 95% CI -47% to 2%).

Multiplicity of results in included studies
Of the 325 studies, 296 studies were included in reviews (n=38) without any prespecified decision rules to select results to include in meta-analyses (Table 6). These reviews, and the studies within, are therefore used to estimate the extent of multiplicity of results that can be expected when no decision rules to select results are predefined. The median (IQR) number of available effect estimates per study was 2 (1 to 3), and the largest number of effect . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint estimates in a study was 41 [62]. The most common types of multiplicity arose from multiple unadjusted and one or more covariate-adjusted analyses (which occurred in 40% of the included studies), followed by multiple intervention/control groups (24%). The least common types of multiplicity arose from multiple instruments (0%). The studies with continuous outcomes had less multiplicity than the studies with non-continuous outcomes (54% vs 80%; RD -26%, 95% CI -40%, -12%).

Discussion
Our findings show that decision rules to select results were infrequently pre-specified in the protocols of a randomly selected sample of SRs with meta-analyses of nutrition research.
Multiplicity of results in the primary studies included in the index meta-analyses was very common, with 69% having at least one type of multiplicity. The frequency and types of multiplicity in the included studies varied, arising from multiple intervention groups, time points, analyses and subgroups.

Comparison with previous research
The findings of our study are in line with previously published studies, which have observed incomplete pre-specification of SR methods such as eligibility criteria, methods for collecting, handling and analysing data, and pre-specification of eligibility criteria and decision rules in PROSPERO records [63], Cochrane protocols [9,64] and published SRs [9,65]. Two previous studies, on which the methodology of the present study is based, examined methods used to select results for inclusion in meta-analyses [12] [9]. Page et al. [9] examined 44 SRs, ,nearly half of which were Cochrane reviews, and they included only randomised trials. In Page et al. [9] and our study, all protocols reported at least one eligibility criteria, but at least one decision rule was pre-specified more in the SR protocols is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint included in Page et al. [9] than those in our study (81% vs 29%). The frequency of discrepancies between the SR protocol and SR in the eligibility criteria or decision rules to select results was higher in our study. Tendal et al. [12] examined eighteen Cochrane reviews, all of which had protocols. Eight of the protocols mentioned eligible time points or periods, but only one provided decision rules to handle multiplicity of time points.
Interestingly, all of the 18 protocols reported eligibility criteria for the control group, but none reported decision rules to handle multiple control groups in included studies.
Three studies (Tendal et al. [12], Mayo-Wilson et al. [13] and Page et al. [9]) assessed the multiplicity of results among the included studies of SRs. Compared to our study, Page et al. [9] and Tendal et al. [12] had fewer studies with multiple estimates that were available for inclusion in a particular meta-analysis. Multiplicity arising from multiple intervention/control groups was slightly less frequent in our study compared to Tendal et al. [12] (24% vs 29%) but was more frequent in ours when compared with Page et al. [9] (24% vs 17%). Our study findings showed less multiplicity in terms of time points (5%) and measurement instruments for outcomes (0%) compared to previous studies [9,12]. However, we observed greater multiplicity (40%) due to unadjusted and one or more covariate-adjusted analyses. This difference was likely driven by inclusion of non-randomized studies in the present study, which often require adjustment for covariates to reduce risk of bias due to confounding [66].

Strengths and limitations
. CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint The major strength of our study is that we have pre-specified methods to identify, select and collect data from eligible SRs and studies, and provided the modifications/deviations to our study protocol (supplementary Table S1). We also used extensive search strategies to identify SRs in nutrition research. In addition, all the study authors, who had different levels of expertise, undertook training and pilot-testing of data collection forms. Moreover, given we randomly selected the SRs, our findings are generalisable to SRs meeting this study's eligibility criteria.
A limitation of our study is that we only retrieved reports of studies included in the index meta-analyses that the systematic reviewers cited. It is possible that other papers relating to the studies exist, which contain additional results that are compatible with the index metaanalysis (e.g. results of cohort studies or randomized trials at later time points may have been presented in other papers which were not cited by the systematic reviewers). For this reason, we may have underestimated the true extent of multiplicity of results within studies. We were unable to translate and interpret the data from four included non-English language studies however, their absence is unlikely to have modified the observed extent of multiplicity.
Finally, we only searched PubMed and Epistemonikos to find all SRs and included SRs written in English, so generalisation of our study findings to non-indexed SRs and non-English language SRs is potentially limited.

Implications of this research for practice
Our study, in common with previous research ( [9,12,13]), demonstrates that multiplicity of effect estimates in primary studies is very common, and is therefore an issue that systematic reviwers should prepare for when designing their review. Doing so will have multiple benefits. It will reduce post-hoc decision making, and in doing-so, provide greater assurance . CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint to a reader that the results have not been 'cherry-picked' for inclusion in the meta-analyses.
Furthermore, specification of eligibility criteria and decision rules is likely to lessen the data extraction effort, requiring less information to be extracted from each primary study.
Our results suggest that in nutrition research, specification of eligibility criteria and decision rules to select results from among multiple unadjusted or covariate-adjusted analyses and overlapping samples of participants and different subgroups are most important to pre-specify in SRs that include non-randomised studies. On the other hand, methods used to select results arising from both final values and change from baseline values, and multiple time points are most important to pre-specify in SRs that include randomized trials with continuous outcomes. Furthermore, in the SR, we recommend reporting any modifications to the specified rules, or any additions that were introduced to cover multiplicity scenarios that had not been anticipated when designing the review.
The Cochrane Handbook for Systematic Reviews of Interventions version 6.1 [67] provides updated guidance to systematic reviewers about how to group interventions with multiple components or co-interventions or how to select from multiple comparisons and handle multiplicity of outcomes when conducting meta-analyses. In addition, the recently updated "Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement" [68] includes a new item (10a), which recommends authors "List and define all outcomes for which data were sought. Specify whether all results that were compatible with each outcome domain in each study were sought (e.g. for all measures, time points, analyses), and if not, the methods used to decide which results to collect". Implementation of recommendations from these sources will allow readers to understand the result selection process. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint

Conclusion
Our study found that in systematic reviews examining the effects of foods and diet, that multiplicity of results in the included primary studies was very common. Yet, prespecification of decision rules to select from multiple results was not common. Systematic reviewers are encouraged to consider methods for dealing with multiplicity when designing their reviews. Doing so will limit the potential for selective inclusion of results, thus providing greater assurance to readers as to the trustworthiness of the review.

Conflicts of interest
All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and have no conflicts of interest to declare.

Contributors
. CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint Table 1: Data sources and data items (see protocol for further details) [14] Source Data items

Systematic review protocol
Year of publication/registration; eligibility criteria and decision rules to select results to include in the index meta-analysis Systematic review General characteristics of the systematic review Journal name; year of publication; corresponding author's country and affiliation; conflicts of interest of review authors; source of funding for the review;

General characteristics of the index meta-analysis
Number of studies and participants; type of population investigated; type of interventions/exposures investigated; type of studies included in the meta-analysis; outcome domain (such as weight, cardiovascular function); outcome primacy label (primary or secondary or unlabelled); meta-analysis effect measure; metaanalysis model; eligibility criteria and decision rules to select results to include in the index meta-analysis; summary statistics, effect estimates and measures of precision (e.g confidence interval) for each included study; and the meta-analytic effect estimate and measure of precision.

Study reports Outcome data that could potentially be included in the index meta-analysis
Outcome definition and measurement instrument; intervention/exposure desciption; comparator description; time point; analysis sample (e.g. intention-to-treat, per-protocol); summary statistics (e.g. number of events and sample sizes of both intervention/exposure and comparator); effect measure (e.g. risk ratio, mean difference); effect estimates and measures of precision (e.g. 95% confidence interval) and location of data in the report; whether results were unadjusted or adjusted for covariates, covariates that were adjusted for (if applicable).
. CC-BY 4.0 International license It is made available under a perpetuity.
is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ;   Other decision rule 5 (12) 1 (5) 4 (17) -12 (-33, 10) *Risk difference calculated as the difference in percentage of SRs reporting the specified eligibility criteria/decision rule between SRs of continuous outcomes minus SRs of non-continuous outcomes. ITT: Intention to treat; NA: Not applicable; **Confidence limits for the difference in percentages calculated using the method of Mee with the Miettinen and Nurminen modification  (5) 13 (10)  is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; *Of the 6259 unique titles and abstracts, we only needed to screen 3,013 randomly sorted titles and abstracts to reach our target sample size. **We initially drew a random sample of 50 systematic reviews, but post-hoc excluded eight systematic reviews with 20 or more included studies in the index meta-analysis to reduce workload. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted August 7, 2021. ; https://doi.org/10.1101/2021.08.05.21261680 doi: medRxiv preprint