RT Journal Article SR Electronic T1 Methodological Challenges using Routine Clinical Care Data for Real-World Evidence: a Rapid Review utilizing a systematic literature search and focus group discussion JF medRxiv FD Cold Spring Harbor Laboratory Press SP 2024.09.05.24313049 DO 10.1101/2024.09.05.24313049 A1 Pfaffenlehner, Michelle A1 Behrens, Max A1 Zöller, Daniela A1 Ungethüm, Kathrin A1 Günther, Kai A1 Rücker, Viktoria A1 Reese, Jens-Peter A1 Heuschmann, Peter A1 Kesselmeier, Miriam A1 Remo, Flavia A1 Scherag, André A1 Binder, Harald A1 Binder, Nadine A1 , YR 2024 UL http://medrxiv.org/content/early/2024/09/06/2024.09.05.24313049.abstract AB Background The integration of real-world evidence (RWE) from real-world data (RWD) in clinical research is crucial for bridging the gap between clinical trial results and real-world outcomes. Analyzing routinely collected data to generate clinical evidence faces methodological concerns like confounding and bias, similar to prospectively documented observational studies. This study focuses on additional limitations frequently reported in the literature, providing an overview of the challenges and biases inherent to analyzing routine clinical care data, including health claims data (hereafter: routine data).Methods We conducted a literature search on routine data studies in four high-impact journals based on the Journal Citation Reports (JCR) category “Medicine, General & Internal” as of 2022 and three oncology journals, covering articles published from January 2018 to October 2023. Articles were screened and categorized into three scenarios based on their potential to provide meaningful RWE: (1) Burden of Disease, (2) Safety and Risk Group Analysis, and (3) Treatment Comparison. Limitations of this type of data cited in the discussion sections were extracted and classified according to different bias types: main bias categories in non-randomized studies (information bias, reporting bias, selection bias, confounding) and additional routine data-specific challenges (i.e., operationalization, coding, follow-up, missing data, validation, and data quality). These classifications were then ranked by relevance in a focus group meeting of methodological experts. The search was pre-specified and registered in PROSPERO (CRD42023477616).Results In October 2023, 227 articles were identified, 69 were assessed for eligibility, and 39 were included in the review: 11 on the burden of disease, 17 on safety and risk group analysis, and 11 on treatment comparison. Besides typical biases in observational studies, we identified additional challenges specific to RWE frequently mentioned in the discussion sections. The focus group had varied opinions on the limitations of Safety and Risk Group Analysis and Treatment Comparison but agreed on the essential limitations for the Burden of Disease category.Conclusion This review provides a comprehensive overview of potential limitations and biases in analyzing routine data reported in recent high-impact journals. We highlighted key challenges that significantly impact analysis results, emphasizing the need for thorough consideration and discussion for meaningful inferences.Competing Interest StatementThe authors have declared no competing interest.Funding StatementThis work was funded by the Federal Ministry of Education and Research (BMBF) in Germany in the framework of the EVA4MII project (FKZ 01ZZ2308A, 01ZZ2308B, 01ZZ2308C). The funding agency had no role in the design, data collection, analyses, interpretation, and reporting of the study. The work of HB, MB, and NB has also been funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - Project-ID 499552394 - SFB 1597.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesI confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.YesAll data produced in the present work are contained in the manuscriptCOPDChronic Obstructive Pulmonary DiseaseEHRElectronic Health RecordsICDInternational Classification of Diseases and Related Health ProblemsJCRJournal Citation ReportOPSOperation and Procedure Classification SystemRCTRandomized Controlled TrialRWDReal-World DataRWEReal-World Evidence