ABSTRACT
Introduction Confounding bias threatens the reliability of observational studies and poses a significant scientific challenge. This paper introduces a framework for identifying confounding factors by exploiting literature-derived computable knowledge. In previous work, we have shown that semantic constraint search over computable knowledge extracted from the literature can be useful for reducing confounding bias in statistical models of EHR-derived observational clinical data. We hypothesize that adjustment sets of literature-derived confounders could also improve causal inference.
Methods We introduce two methods (semantic vectors and string-based confounder search) that query the literature for potential confounders and use this information to build models from EHR-derived data to more accurately estimate causal effects. These methods search SemMedDB for indications TREATED BY the drug that is also known to CAUSE the adverse event. For evaluation, we attempt to rediscover associations in a publicly available reference dataset containing expected pairwise relationships between drugs and adverse events from empirical data derived from a corpus of 2.2M EHR-derived clinical notes. For our knowledge-base, we use SemMedDB, a database of computable knowledge mined from the biomedical literature. Using standard adjustment and causal inference procedures on dichotomous drug exposures, confounders, and adverse event outcomes, varying numbers of literature-derived confounders are combined with EHR data to predict and estimate causal effects in light of the literature-derived confounders. We then compare the performance of the new methods with naive (χ2, reporting odds ratio) measures of association.
Results and Conclusions Logistic regression with ten vector space-derived confounders achieved the most improvement with AUROC of 0.628 (95% CI: [0.556,0.720]), compared with baseline χ20.507 (95% CI: [0.431,0.583]). Bias reduction was improved more often in modeling methods using more rather than less information, and using semantic vector rather than string-based search. We found computable knowledge useful for improving automated causal inference, and identified opportunities for further improvement, including a role for adjudicating literature-derived confounders by subject matter experts.
Highlights
Access to causal background knowledge is required for causal learning to scale to large datasets.
We introduce a framework for identifying confounders to enhance causal inference from EHR.
We search computable knowledge for indications TREATED BY the drug that CAUSE the outcome.
Literature-derived confounders reduce confounding bias in EHR data.
Structured knowledge helps interpret and explain data captured in clinical narratives.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This research was supported by the US National Library of Medicine grants: R01 LM011563, 2 T15 LM007093$-$26, 5 T15 LM007059$-$32, NIH/BD2K supplement R01 LM011563$-$02S1, NCATS Grant U54 TR002804, Cancer Prevention Research Institute of Texas (CPRIT) Precision Oncology Decision Support Core RP150535, and CPRIT Data Science and Informatics Core for Cancer Research (RP170668).
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
University of Texas and University of Pittsburgh IRBs.
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
Following IRB approval and a data usage agreement, we obtained permission to use the EHR data for this study. These data require approval to access. However, we have included the models and other analytic data from this project.
