Skip to main content
medRxiv
  • Home
  • About
  • Submit
  • ALERTS / RSS
Advanced Search

Tool to assess risk of bias due to missing evidence in network meta-analysis (ROB-MEN): elaboration and examples

View ORCID ProfileVirginia Chiocchia, Adriani Nikolakopoulou, Julian PT Higgins, Matthew J Page, Theodoros Papakonstantinou, Andrea Cipriani, Toshi A Furukawa, George CM Siontis, Georgia Salanti
doi: https://doi.org/10.1101/2021.05.02.21256160
Virginia Chiocchia
1Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
2Graduate School for Health Sciences, University of Bern, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Virginia Chiocchia
  • For correspondence: virginia.chiocchia@ispm.unibe.ch
Adriani Nikolakopoulou
1Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
3Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Julian PT Higgins
4Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Matthew J Page
5School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Theodoros Papakonstantinou
1Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
3Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Germany
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Andrea Cipriani
6Department of Psychiatry, University of Oxford, Oxford, UK
7Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Toshi A Furukawa
8Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine and School of Public Health, Kyoto, Japan
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
George CM Siontis
9Department of Cardiology, Bern University Hospital, Inselspital, Bern, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Georgia Salanti
1Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Abstract
  • Full Text
  • Info/History
  • Metrics
  • Supplementary material
  • Data/Code
  • Preview PDF
Loading

Abstract

Selective outcome reporting and publication bias threaten the validity of systematic reviews and meta-analysis and ultimately can affect clinical decision-making. A rigorous methodology to evaluate the impact of this bias on the meta-analysis results of a network of interventions is still lacking. We present a tool to assess the Risk Of Bias due to Missing Evidence in Network meta-analysis (ROB-MEN) by expanding the methods previously developed for pairwise meta-analysis (ROB-ME, http://www.riskofbias.info).

ROB-MEN first evaluates the risk of bias due to missing evidence for each pairwise comparison separately. This step considers possible bias due to the presence of studies with unavailable results (known unknowns) and the potential for unpublished studies (unknown unknowns). The second step combines the overall judgements about the risk of bias due to missing evidence in pairwise comparisons with the percentage contribution of direct comparisons on the NMA estimates, the presence or absence of small-study effects, as evaluated by network meta-regression, and any bias from unobserved comparisons. Then, a level of “low risk”, “some concerns” or “high risk” for the bias due to missing evidence is assigned to each NMA estimate, which is our tool’s final output.

We describe the methodology of ROB-MEN step-by-step using an illustrative example from a published NMA of non-diagnostic modalities for the detection of coronary artery disease in patients with low risk acute coronary syndrome. We also report a full application of the tool on a larger and more complex published network of 18 drugs from head-to-head studies for the acute treatment of adults with major depressive disorder. The ROB-MEN tool is the first tool for evaluating the risk of bias due to missing evidence in NMA and it is applicable to networks of all sizes and geometry.

Competing Interest Statement

All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: AC has received research and consultancy fees from INCiPiT (Italian Network for Paediatric Trials), CARIPLO Foundation and Angelini Pharma; TAF reports personal fees from MSD, grants and personal fees from Mitsubishi-Tanabe, grants and personal fees from Shionogi, outside the submitted work; TAF has a patent 2018-177688 pending, and a patent Kokoro-app issued; no other relationships or activities that could appear to have influenced the submitted work.

Funding Statement

The development of the ROB-MEN web application and part of the presented work was supported by the Cochrane Collaboration. GS, VC,AN and TP are supported by project funding (Grant No. 179158) from the Swiss National Science Foundation (SNSF). AN is supported by a SNSF personal fellowship (P400PM_186723). JPTH is a National Institute for Health Research (NIHR) Senior Investigator (NF-SI-0617-10145) and is supported by the National Institute for Health Research (NIHR) Bristol Biomedical Research Centre at University Hospitals Bristol and Weston NHS Foundation Trust and the University of Bristol, NIHR Applied Research Collaboration West (ARC West) at University Hospitals Bristol and Weston NHS Foundation Trust and NIHR Health Protection Research Unit in Evaluation of Interventions at the University of Bristol in partnership with Public Health England. MJP is supported by an Australian Research Council Discovery Early Career Researcher Award (DE200101618). C is supported by the National Institute for Health Research (NIHR) Oxford Cognitive Health Clinical Research Facility, by an NIHR Research Professorship (grant RP-2017-08-ST2-006), by the NIHR Oxford and Thames Valley Applied Research Collaboration and by the NIHR Oxford Health Biomedical Research Centre (grant BRC-1215-20005). The views expressed in this article are those of the authors and do not necessarily represent those of the SNSF, NHS, the NIHR, MRC, or the Department of Health and Social Care.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

Not applicable

All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Yes

Footnotes

  • Competing interest statement updated.

Data Availability

Data sharing not applicable as no datasets generated and/or analysed for this study.

Copyright 
The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license.
Back to top
PreviousNext
Posted May 06, 2021.
Download PDF

Supplementary Material

Data/Code
Email

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Enter multiple addresses on separate lines or separate them with commas.
Tool to assess risk of bias due to missing evidence in network meta-analysis (ROB-MEN): elaboration and examples
(Your Name) has forwarded a page to you from medRxiv
(Your Name) thought you would like to see this page from the medRxiv website.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Tool to assess risk of bias due to missing evidence in network meta-analysis (ROB-MEN): elaboration and examples
Virginia Chiocchia, Adriani Nikolakopoulou, Julian PT Higgins, Matthew J Page, Theodoros Papakonstantinou, Andrea Cipriani, Toshi A Furukawa, George CM Siontis, Georgia Salanti
medRxiv 2021.05.02.21256160; doi: https://doi.org/10.1101/2021.05.02.21256160
Reddit logo Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools
Tool to assess risk of bias due to missing evidence in network meta-analysis (ROB-MEN): elaboration and examples
Virginia Chiocchia, Adriani Nikolakopoulou, Julian PT Higgins, Matthew J Page, Theodoros Papakonstantinou, Andrea Cipriani, Toshi A Furukawa, George CM Siontis, Georgia Salanti
medRxiv 2021.05.02.21256160; doi: https://doi.org/10.1101/2021.05.02.21256160

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Subject Area

  • Epidemiology
Subject Areas
All Articles
  • Addiction Medicine (239)
  • Allergy and Immunology (521)
  • Anesthesia (124)
  • Cardiovascular Medicine (1418)
  • Dentistry and Oral Medicine (217)
  • Dermatology (158)
  • Emergency Medicine (291)
  • Endocrinology (including Diabetes Mellitus and Metabolic Disease) (582)
  • Epidemiology (10288)
  • Forensic Medicine (6)
  • Gastroenterology (527)
  • Genetic and Genomic Medicine (2625)
  • Geriatric Medicine (254)
  • Health Economics (496)
  • Health Informatics (1729)
  • Health Policy (789)
  • Health Systems and Quality Improvement (671)
  • Hematology (266)
  • HIV/AIDS (564)
  • Infectious Diseases (except HIV/AIDS) (12083)
  • Intensive Care and Critical Care Medicine (648)
  • Medical Education (273)
  • Medical Ethics (83)
  • Nephrology (288)
  • Neurology (2456)
  • Nursing (144)
  • Nutrition (377)
  • Obstetrics and Gynecology (491)
  • Occupational and Environmental Health (566)
  • Oncology (1320)
  • Ophthalmology (400)
  • Orthopedics (146)
  • Otolaryngology (235)
  • Pain Medicine (168)
  • Palliative Medicine (51)
  • Pathology (342)
  • Pediatrics (778)
  • Pharmacology and Therapeutics (329)
  • Primary Care Research (296)
  • Psychiatry and Clinical Psychology (2395)
  • Public and Global Health (4999)
  • Radiology and Imaging (893)
  • Rehabilitation Medicine and Physical Therapy (525)
  • Respiratory Medicine (681)
  • Rheumatology (309)
  • Sexual and Reproductive Health (255)
  • Sports Medicine (244)
  • Surgery (297)
  • Toxicology (45)
  • Transplantation (140)
  • Urology (108)