A systematic review of ontology-based clinical decision support system rules: usage, management, and interoperability

Xia Jing, Hua Min, Yang Gong, Dean F. Sittig, Paul Biondich, David Robinson, Tim Law, Adam Wright, Christian Nøhr, Arild Faxvaag, Lior Rennert, Nina Hubig, Ronald Gimbel
doi: https://doi.org/10.1101/2022.05.11.22274984

Abstract

Objective Clinical decision support systems (CDSS) have a critical role in improving the quality and safety of health care delivery. CDSS rules direct the behavior of CDSS. However, the CDSS rules have not been routinely shared and reused, and ontology can promote the reusing of CDSS rules. We systematically screened literature to elaborate on the current status of ontology applied in CDSS rule management.

Methods We searched PubMed, the Association for Computing Machinery (ACM) Digital Library, and the Nursing & Allied Health Database for publications focusing on ontology, clinical decision support, and rules. Grounded theory and PRISMA 2020 guidelines were followed. One author started the screening and literature analysis, and two authors validated the processes and results. Inclusion and exclusion criteria were developed and refined iteratively.

Results Among 81 included publications, the identified CDSS were mainly applied to managing chronic conditions, alerts for medication prescriptions, reminders for immunizations and preventive services, diagnoses, and treatment recommendations. The CDSS rules were presented in Semantic Web Rule Language, Jess, or Jena formats. Despite ontology was used to supply medical knowledge, CDSS rules, and terminologies to CDSS, ontology has not been used in CDSS rule management.

Conclusions Although ontology can facilitate the reuse, management, and maintenance of CDSS rules, CDSS ontology remains unavailable indicating that more efforts are needed to improve the reusability and interoperability of CDSS rules.

Introduction

Clinical Decision Support Systems (CDSS) have been studied and used in clinical care delivery for longer than half a century [1–5]. The effectiveness of CDSS in clinical care has been established [6–8]. We cite some prominent examples in CDSS. Middleton et al. reviewed the two decades of clinical decision support experiences as CDSS users and developers and discussed their vision of CDSS for the future [9]. Wright et al. recommended best practice guidelines in CDSS [10–14]. Meanwhile, Sittig and colleagues comprehensively documented the challenges of CDSS [15]. McDonald is one of the earliest pioneers who demonstrated the effectiveness of CDSS in clinical care [16]. Meeting clinicians’ information needs is one way to improve clinical care outcomes via CDSS. Many studies have shown the effectiveness of such CDSS, for example, Cimino’s Infobutton [6, 17]. Currently, CDSS is used routinely in clinical care in the USA. An analysis of a 2015 national survey showed CDSS usage rates reached 68.5% to 100% in primary care settings based in offices [18] in the USA. Therefore, CDSS is commonly used in clinical care delivery as part of electronic health record (EHR) systems. CDSS has a range of forms: such as reminders for preventive services (e.g., immunizations, screening tests) [19, 20], alerts for drug-drug interactions [14, 21, 22], diagnostic or treatment plan recommendations [23–25], content assistance for clinicians [26–30], or recommendations for adhering to current clinical practice guidelines [31–33]. CDSS has played a critical role and is used commonly in providing safer and better quality clinical care services.

CDSS rules, like central neural systems for humans, direct the behaviors of a CDSS during operations by incorporating a patient’s data, contextual information, and medical domain knowledge. CDSS rules have a central role and are a decisive factor in the relevance and usefulness of CDSS in the entire clinical workflow, including whether a CDSS will be adopted and used. CDSS rules can be written in different languages, such as Arden syntax [34], the Semantic Web Rule Language (SWRL), Jess, Jena, and other programming languages. Reusing and sharing CDSS rules is ideal since it is very costly to recreate these rules every time, but the process is not a routine operation across organizations. As a component of an EHR system, updating the CDSS rules regularly is needed to keep the CDSS relevant and useful in clinical care delivery. However, it is a very time-consuming and resource-demanding process to create, manage, maintain, reuse, and share CDSS rules—one type of CDSS artifact. This resource-demanding process has been recognized by larger institutes [4, 35], which usually are better resourced than small-scale practices. Therefore, creating and maintaining CDSS rules can be much more challenging in resource-limited office-based practices.

The ontology can facilitate the use, reuse, sharing, and interoperability of CDSS rules, solving the challenges of managing and maintaining CDSS rules across institutional borders. Ontology is the enabling technology of the Semantic Web [36] and is critical in information sharing and reusing [37, 38]. Although there are many definitions of ontology, we use the one by Gruber: “an ontology is a specification of conceptualization” [37]. Ontology has been used to generate and supply domain knowledge in many fields, including medicine [39] and CDSS [40], which is not an ontology that focuses on CDSS but includes relevant content of CDSS. Ontology can be used to shed light on achieving interoperable clinical information, such as CDSS rules, at a more granular level.

Although ontology can facilitate the reuse and sharing of CDSS rules, no CDSS ontology exists. Therefore, we are building a CDSS ontology [41]. Furthermore, to elaborate on the current status of using ontology in CDSS rules, to improve CDSS rules reuse and interoperability, we systematically reviewed existing English literature about the intersection of CDSS rules, Semantic Web technology (especially ontology), and the use of ontologies in CDSS. The primary rationale of this systematic literature review is that even though there is no existing CDSS ontology except for our effort, the systematic examination of the literature on the intersection of CDSS rules, ontology, and the use of ontology in CDSS can provide a clearer picture on how ontology is used in CDSS, especially regarding CDSS rules. The literature review can shed light on our effort to use the Semantic Web technology to enable the reuse and maintenance of CDSS rules.

Through this effort, we aimed to achieve three objectives. First, this systematic review will provide an initial structure and considerations for organizing our CDSS ontology. Second, the results can help us hone our focus area: ontology use in CDSS rule management. A clear scope and comprehensive picture can form a knowledge framework of the topic that may inspire and guide future efforts to advance this field more explicitly. Third, the manually annotated results of selected publications will be used as gold standards to automatically identify entities from the literature, which complements efforts to curate CDSS ontology manually.

Methods

General workflow

After the literature search, the first 100 of all the retrieved papers were initially screened by one author (XJ), and initial inclusion and exclusion criteria were drafted. This screening was replicated, and the results were discussed, validated, and the criteria were refined and adjusted by two other authors (HM and YG). Therefore, the first 100 papers were screened by all three authors independently. The inclusion and exclusion criteria were developed, revised, and refined during the iterative literature screening, review, analysis, and discussions. Then the rest of the papers were screened by at least two authors (XJ and MH, or XJ and YG) independently to determine if a paper should be included in the literature review. The discrepancies were discussed and resolved via iterative rounds of teleconferences.

When needed, the literature was first screened using publication titles, abstracts, and full-text publications. Then after determining which papers to include, included papers were manually coded to provide more content analysis and synthesized evidence. The final results were shared and agreed upon among all authors. Therefore, the screening of each paper and manual review of included papers were conducted by two authors independently and approved by at least two authors first and all authors later. All discrepancies and concerns were resolved via group email discussions and complementary teleconferences. Figure 1 illustrates the general workflow.

Figure 1
Figure 1

General workflow of the systematic literature review

Databases and search strategies

On June 2, 2020, we did an initial set of literature searches. After initial review and discussions, we refined and agreed with the search strategies. On Sept 15, 2020, we searched PubMed, the Association for Computing Machinery (ACM) Digital Library, and the Nursing & Allied Health Database for literature with the following search strategies. On January 5, 2022, we conducted the exact search again in the three literature databases as an update.

  • PubMed

(clinical decision support systems[MeSH Terms]) AND (ontolog*[Title/abstract] OR rule*[Title/abstract])

  • ACM Digital Library

The following search was conducted within the scope of the ACM Guide to Computing Literature.

[[Publication Title: “clinical decision support*”] OR [Publication Title: cds*]] AND [[Publication Title: ontolog*] OR [Abstract: ontolog*] OR [Publication Title: rule*] OR [Abstract: rule*]]

  • Nursing & Allied Health Database (NAHD)

The following search was limited to peer-reviewed publications.

mesh(clinical decision support) AND (ti(ontology) OR ti(ontologies) OR ab(ontology) OR ab(ontologies) OR ti(rule) OR ti(rules) OR ab(rule) OR ab(rules))

Inclusion and exclusion criteria

The inclusion criteria were:

  1. Literature is written in English;

  2. Full-text publication available;

  3. Ontology designed to implement or implemented in CDSS, especially related to CDSS rules;

  4. Content includes granularity of CDSS rules;

  5. Ontology designed to integrate or already integrated with health information systems (e.g., EHR), either in a production system or a prototype, with at least one architecture diagram;

  6. Applied in clinical domains or designed for clinical domains to support health care providers;

  7. Peer-reviewed publication; and

  8. Contained details on integration between CDSS and EHR for evaluation studies.

The exclusion criteria were:

  1. Only CDSS rules, regardless of any stage of the CDSS rules’ lifecycle (i.e., development, identification, refinement, validation, evaluation, or implementation), or no integration was mentioned, or no mention of ontology;

  2. Only Ontology was developed, evaluated, and validated, or no integration was mentioned, or no CDSS was mentioned;

  3. System was designed without mentioning the granularity of CDSS rules or ontology; and

  4. The publication described nonclinical decisions, such as administrative or management decisions (e.g., supply chain management).

Reviewing, coding, analyzing, and synthesizing processes

During the reviewing and manual coding, we followed grounded theory. One author (XJ) randomly selected ten papers from the included 81 papers to start the coding based on the focus of this literature review: CDSS rules (including application and management), ontology, and roles of ontology. The coding results were discussed by three authors (XJ, HM, and YG). The discussion results formed the first draft of codes and code groups (Appendix 1), i.e., data items. Then the three coders (authors) reviewed and coded the first 40 of the included papers using the initial principles and code groups and adding new codes and code groups when needed. Midway through coding, consensus on updated principles and code groups was obtained at a second meeting and following iterative discussions. Then the refined codes and code groups were used to code the remaining papers. Every paper was coded by at least two coders independently. Then the coding results were compared, and any discrepancies were discussed to resolve. During each discussion, the code groups and codes were revised, consolidated, and updated. The refined code groups and examples are described in Appendix 2. Therefore, the data items were emerged during review and refined via discussions instead of pre-defined before reviewing.

A qualitative data analytic tool-ATLAS.ti 9 (desktop version and Web version), was used for coding by the three coders. To form the final codes and code groups, all projects (ATLAS.ti organizes the data as projects, i.e., each coder has a project to save all the papers and coding results) from all three coders were merged, and any duplicates were removed after a manual check. Then codes and code groups were later combined, and any duplicates were removed after a manual review.

After coding, the literature was analyzed and synthesized. For this process, we focused on several aspects, including the application domains of CDSS, the mechanism of CDSS used in clinical settings, CDSS rule format, authoring, management, and ontology roles. The process was iterated several times among the three coders first. Then after obtaining consensus among the three coders, the results were shared and discussed among all authors later. Through iterative discussions, any concerns, confusion, or disagreements among authors were resolved with group emails and complementary teleconferences.

We have followed the PRISMA 2020 checklist [42] in reporting the systematic review with all relevant items.

Results

Results related to CDSS characteristics

The literature searches generated 1,235 publications from three sources until Jan 5, 2022. After removing duplicates and examining according to inclusion and exclusion criteria, we included 81 publications (Appendix 3) in the final review and analysis [19-21, 23-25, 43-117]. Figure 2 illustrates each step of the literature search, screening, selection flow, and results. Figure 3 summarizes the main components covered by the literature review and the summary findings. Figure 3 also serves as an initial knowledge framework on CDSS, CDSS rules, and ontology applications in CDSS.

Figure 2
Figure 2

Flowchart of the literature search, screening, and selection

Figure 3
Figure 3

Initial knowledge framework on CDSS, CDSS rules, and ontology applications in CDSS

More than one-third (35.8%, 29/81) of CDSS was used for chronic condition management, prediction, or risk assessment. The chronic conditions included type 1 and 2 diabetes, hypertension, and asthma. Other significant domains included medication prescriptions (16%; 13/81), such as medication orders and detection of adverse drug events and drug-drug interactions, and cancer care (9.9%; 8/81). Most CDSS was designed and used by health care providers, and only 11.1% (9/81) were intended to be used by patients, some of which can be used by clinicians too. Most CDSS provide recommendations, suggestions, alerts, or reminders. Among all items we compared (Table 1), EHR evaluation had the least complete information. Some CDSS was implemented in production systems (38.3%; 31/81), excluding design, and some were implemented in prototypes (37%; 30/81), including experimental systems. Table 1 summarizes the essential characteristics of CDSS. We used the terms from the corresponding papers in all tables without changing them. For example, some papers used “physicians” as CDSS users, and others used “clinicians” as CDSS users. We used the authors’ terms in the tables.

View this table:
Table 1

Basic CDSS profiles in included publications (n = 81)

Results related to CDSS rules

Most CDSS rules were written in the languages of SWRL (11.1%; 9/81), Web Ontology Language (OWL; 13.6%, 11/81), extensive markup language (XML; 12.3%; 10/81), Jena rules (6.2%; 5/81), and medical logic module (MLM; 3.7%; 3/81). Two publications [109, 111] used N3 language, and two [82, 109] used the natural rule language (NRL). Table 2 details the CDSS rules for publications that can fill out three (except for the authors and publication years) or more cells in table 2 (n = 54).

View this table:
Table 2

Comparison of CDSS rule characteristics in included publications (n = 54)

The most significant CDSS rule sources were from clinical practice guidelines (44.4%; 36/81). Other sources include domain expert input, publications (e.g., textbooks, papers), multimedia sources, and Internet resources. Two publications used data mining results as CDSS rule sources [59, 65]. CDSS rule authoring and editing tools were not routinely specified in the publications. Protégé [115] was the most popular tool to edit and author CDSS rules in publications with such detail. Several publications also describe developing such tools [49, 57, 83].

Rule engines are another technical detail that many publications did not specify. Among all publications with specified rule engines, Jena (7.4%; 6/81), inference engine (7.4%; 6/81), Jess (4.9%; 4/81), JBoss (3.7%; 3/81), guideline engine (3.7%; 3/81), Drools (2.5%; 2/81), and Bayes (2.5%; 2/81) were most often used. The column for CDSS rule operation in Table 2 summarizes how the CDSS rule works in a simplified manner. Many publications do not provide details on the working mechanism of CDSS rules within the system context.

Interoperability does not seem to be the focus of the publications. Only a few papers mentioned interoperability (Table 2). They all used HL7 CDA (Health Level 7, Clinical Document Architecture) or HL7 FHIR (Fast Healthcare Interoperability Resources) standards. However, we noticed that such measures are not specifically for CDSS rules but mainly for the input and output of CDSS.

Results related to ontology

In the included publications, ontology is mainly used as a knowledge source for CDSS (39.5%; 32/81) and to facilitate classification (8.6%; 7/81), reasoning, and inference (7.4%; 6/81; e.g., identification recommendations or relationships). Ontologies were also used to specify CDSS rules (15%; 12/80) or provide general knowledge for the electronic medical record (EMR) or electronic health record (EHR) systems. In some cases, these two uses also overlapped (23.5%; 19/81; i.e., the ontologies were used to provide specified CDSS rules and general knowledge).

Many authors use reasoner and rule engines interchangeably. In this manuscript, we distinguish the two concepts in a workable manner to use them. We do not intend to define them universally and comprehensively. In our description, we use the term reasoner to refer to the inference for a consistency check or classification for an ontology, which can be part of an ontology tool or external. In our description, we use the term rule engine to refer to generating or providing recommendations by incorporating a patient’s data, contextual information, and medical knowledge (usually from an ontology or knowledge base) for CDSS. However, we used the authors’ choice of terms in tables without modification. Among the publications, the most common reasoners were Pellet (13.6%; 11/81), Jena (4.9%; 4/81), the OWL reasoner (3.7%; 3/81), Jess (2.5%; 2/81), and the Euler/EYE inference engine (2.5%; 2/81).

Ontology sources should include the content and code systems used to represent the content. Content may come from a popular textbook or clinical practice guideline. The content can be coded in a specific code system, such as SNOMED CT. Because the CDSS rule source has been specified in Table 2, we only included code systems that served as a source of ontology in Table 3; We used this format to avoid redundancy in the presentation. The most often used coding systems were SNOMED CT (11.1%; 9/81), ICD10 (the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision; 4.9%; 4/81), Unified Medical Language System (UMLS; 4.9%; 4/81), Logical Observation Identifiers Names and Codes (LOINC; 3.7%; 3/81), RxNorm (2.5%; 2/81).

View this table:
Table 3

Comparison of ontology roles in included publications (n = 36)

Many included publications also did not include complete information for ontology validation. Approximately 20 publications mentioned some validation, including validation or evaluation by domain experts (24.7%; 20/81). Some ontologies were authored by domain experts [47, 55]. Table 3 shows additional details on the roles of ontology in the publications, although Table 3 includes only the publications that can fill out three (except for the authors and publication years) or more cells (n = 36).

Other results

After cleaning, discussion, and consolidation, we had 30 code groups and 221 final codes used in the ATLAS.ti (Appendix 2). These codes and code groups guided our results analysis and synthesis; these codes and code groups also helped us form the starting structure for the ontology we are building [41], i.e., the ontology for CDSS rule management and maintenance. Figure 4 illustrates the clinical domains of CDSS within included publications. Figure 5 shows a word cloud image generated from ATLAS.ti based on the codes used in all included publications. In the publications, we also found many large international collaborations [94, 101, 110] and solo author publications [57, 64, 105, 106]. We also noticed that some publications were missing necessary information for explaining the mechanisms of the systems that can be critical for reproducibility. Some examples include the architecture diagrams of CDSS, CDSS rule engine, CDSS rule language, how CDSS rules are managed at the backend, and integration mechanisms among CDSS rules, ontology, and EHR/EMR systems.

Figure 4
Figure 4

Bar chart generated from ATLAS.ti showing the clinical domains of CDSS in included publications.

Figure 5
Figure 5

Word cloud generated from ATLAS.ti based on codes in the included publications

Based on our review, PubMed is still the clear dominant source to supply publications in this field. Approximately 90.1% (73/81) of publications were identified via PubMed. The ACM Library added only eight publications after removing duplicates.

We have followed the PRISMA 2020 checklist in writing this systematic review. However, we realize that PRISMA 2020 is intended to be used by outcome-oriented studies, which is not the focus of our systematic literature review. The publications included in our review are about design, development, and implementation. Therefore effect measures or certainty assessment were irrelevant items. We reported 18 items (out of 27 items, Appendix 4) in the full-text paper and ten in the abstract (out of 12 items, Appendix 5).

Discussion

Summary of results

Although CDSS has proved its effectiveness in clinical care for decades, the reusing and sharing of machine-processable CDSS rules has not been achieved. Ontology has the potential to facilitate interoperable CDSS rules. We conducted this systematic review on CDSS rules, ontology, and its applications in CDSS. Our systematic review shows the broad clinical application domains of CDSS in chronic condition management, medication order, and cancer care. Most CDSS is designed for health care providers, and most of the CDSS rules are based on clinical practice guidelines. Although we found details about CDSS rules, we noticed inconsistent presentations or unavailable information on CDSS rule language, CDSS rule engine, and evaluation details. We also observed inconsistent technical details related to ontology purposes, reasoners, mechanisms of connection, or communication with CDSS or EMR/EHR/HIS. Although some publications describe such a level of detail, not all publications include this information. It can be hard to perform reproducibility and further improvement without such information.

Although reusing and sharing CDSS artifacts is a challenge with very good recognition [1, 101], the reusability, customization, and shareability of CDSS rules are not yet a common focus, not even in publications that focus on CDSS rule editing [35, 118, 119]. We feel these topics are important, especially using ontology to achieve the goals. Therefore we conducted this literature review. Marco-Ruiz et al. [101] showed how to use CDSS artifacts in the Linked Data framework [120] by leveraging Semantic Web technologies, especially ontology. However, that work was at a higher level to describe the concepts without tangible tools implemented in clinical practices. Our motivation to build a CDSS ontology [41] aligns well with their vision. We are trying to shorten the gap between the high-level vision on these realistic issues (reusing and sharing CDSS artifacts) [1, 101] and CDSS operations in the clinical world by appraising existing efforts and using a more generic approach. The first step in filling this gap is constructing an upper-level CDSS ontology.

Significance of the work

We systematically reviewed literature about ontology applied in CDSS. Our approach illustrated the state-of-the-art applications of ontology in CDSS rule management. These applications present excellent potential in reusing and sharing CDSS artifacts, which are critical to clinical care quality and patient safety. Currently, it is very costly to create, maintain, and reuse these rules. Although some authors recognized this benefit [1, 35, 101], none have conducted a systematic review. Our literature review thoroughly analyzed the topic, established the knowledge framework, and created a comprehensive collection that can inform efforts to design or improve future CDSS, create a tutorial on the subject for newcomers in the field, and set the foundation for the CDSS ontology we are building.

We also recommend that authors include essential technical details in publications focusing on this topic. These details include CDSS application domain, CDSS users, CDSS notification types, CDSS evaluation (what, how, by whom), CDSS rule sources, CDSS rule language, CDSS rule engine, CDSS operation mechanism, ontology use purposes, ontology sources (both content and code systems), ontology validation, reasoner, connection/communication mechanism with CDSS, and EMR/EHR. We encourage authors to include such details to help readers reference, compare, and increase the reproducibility of the reported work.

Interpretation of the results

Rule engines are critical components of CDSS. The rule engine executes rules and patient data and context information to produce a result, e.g., an alert or a recommendation [1]. Jess is a rule engine and development environment in Java [121]. We noticed in our review that Jess is one of the famous rule engines used repeatedly. Jess can be used to develop rule-based expert systems or create Jess rules. In the popular tool Protégé, the plug-in API SWRLJessTab can convert SWRL rules to Jess rules. Jess rules can be used by Jess Rule Engine, which has many implementations in rule-based expert systems [121]. In addition to Jess, Jena and Drools were used repeatedly among the included publications. Jena is an API in Java designed to support rule-based inference and use RDF (Resource Description Framework) graphs [122]. Jena.java API can handle OWL models and is a popular framework for managing RDF/OWL descriptions [88]. Drools is a business rule management system, including a rule engine [123]. Drools also have SWRL API, which can support SWRL and SQWRL. SQWRL can query SWRL.

Reasoning via a reasoner is a critical characteristic for many ontologies, even though the current reasoning is still in the first-degree logic. Reasoning can be used for three main functions: consistency check, classification, and realization [124]. Several publications included in this systematic review specified the classification roles of the ontology and the reasoners (listed in Table 3). The OWL group has curated an updated list of OWL reasoners at the University of Manchester in the UK [124]. Parsia and colleagues compiled and compared the current OWL reasoners and their performances via the competition report [124]. Here, we only briefly discuss the reasoners used in the included publications.

Among the included publications, both Pellet and Jena are popular reasoners (Table 3), and other used reasoners include FaCT++ [90], Z3 Solver reasoner [97], Euler/EYE inference engine [109, 111], OWL Horst [101], and OWL Cerebra [55]. Among these reasoners, Pellet [125] is Java-based, and it can work on SWRL rules and ontologies written in OWL2. SWRL was initially designed as a rule language for Semantic Web technologies [126]. A user needs the rule language and an editor (e.g., Protégé SWRL tab) to write, revise, and query the rules. SWRL can be queried by SQWRL (a query language for OWL) or SPARQL (SPARQL Protocol and RDF query language). Then reasoners can be used to conduct reasoning based on the rules and facts defined in the ontology or knowledge base. Protégé-OWL [127] provides an editor for SWRL rules. Protégé SWRL editor is another example.

Not all ontologies or knowledge bases were formally evaluated, validated, or authored by domain experts. Only 25% of publications mentioned about evaluated or validated by domain experts. A formal assessment or validation is critical, regardless of a manually built ontology (or knowledge base). For some ontologies (or knowledge bases) derived from other automatic methods (e.g., machine learning algorithms), the validation is even more critical to ensure the validity of the results from automated processes. Testing has not been conducted consistently across the publications. Some have been formally tested with patient databases in the production systems, some were tested via prototypes, and some were tested conceptually. Technically, all such testing is valuable, to some extent. Without some types of testing, the validity of the work can be questionable. Some ontologies were authored by domain experts, which provides greater validity than those involving nondomain experts while constructing the ontology.

Also, not all publications clearly state whether the evaluation was conducted on the CDSS. For those publications with CDSS evaluation, not all provide adequate details on what was evaluated, how the evaluation was completed, and who the evaluators were.

We noticed that not all included publications provided the needed technical details during the literature review, such as the CDSS rule engine, rule language, and ontology reasoner. Some people may argue that such information can vary dramatically during implementation. However, we believe that including such details will present the work more robustly, help users better understand the system’s mechanism, provide possibilities for others to reproduce the work, and enable others to compare the technical solutions and design more optimal options in the future.

One systematic review by Dissanayake and colleagues [40] was similar to our review. However, our review has several distinct characteristics that complement the literature with unique insights about CDSS rules, ontology, and ontology applications, especially in rule management. For example, we focused on the mechanism of these characteristics in the HIS/EHR/EMR systems. In contrast, Dissanayake’s review focused on the reasoning processes during clinical decision-making, especially in forming the ontologies to support complex cognitive processes. Their study also compared evaluation metrics and evidence generated among the included publications, whereas we focused on the CDSS rule authoring, management, implementation, and ontology validation.

Our coding revealed factual evidence without any interpretations, which differs from qualitative data analysis and coding. Our discussions focused on the scope of coding, definitions of codes, the granularity of coding, dimensions, organization, categorization, and combinations of codes, articulating these points and obtaining consensus.

Knowledge gap and challenges moving forward

Although the importance and challenging nature of reusability and sharability of CDSS artifacts have been well recognized and documented, various efforts are focusing on this topic because of the potential of the Semantic Web technologies and ontology [1, 101]. However, the steps are mainly at the conceptual level without tangible tools to implement in the real world. No CDSS ontology bridges the high-level thinking, planning presented by different groups, and real-world implementations and operations of the process. This knowledge gap motivated our efforts to build a CDSS ontology that focuses on CDSS rule management and maintenance [41]. Moving forward, one challenge is more on the marketing side: let CDSS artifacts developers, vendors, and healthcare professionals, i.e., end-users, be aware of the CDSS ontology and adopt it in their work.

Strengths

This systematic review focuses on CDSS rules and ontology roles in CDSS and mechanisms of CDSS in clinical practices or prototypes. We established the initial knowledge framework on the intersections of CDSS, CDSS rules, and ontology. We summarized detailed information regarding the reasoner, rule engines, ontology, and CDSS rule formats used in each included publication. We also provided summaries (Tables 1–3) of valuable references for designing or improving systems. The side-by-side comparison also gives structured guidance for preparing future publications. We hope the knowledge framework, detailed summarizations, and comparisons can guide and enlighten future improvements and designs more explicitly in tangible ways.

Limitations

We recognize that our review has limitations. We were unable to include publications written in a language other than English. Also, we only included publications with full-text available. We had to exclude one thesis [128], which was a search result from the ACM Library, due to the unavailability of the full-text publication. Also, we excluded some publications that focused on the CDSS rule [35, 118, 119]. Although such publications had a similar focus to one aspect of our systematic review, none specified an ontology component in the publications. We also noticed that the publications on CDSS rule authoring and managing tools are from Partners HealthCare/Harvard Medical School.

We also exclude another category of papers that discussed best practices in CDSS rule management, such as Wright A et al. [10]. Although such publications can inform CDSS design, testing, and operation processes, they did not meet the focused criteria for this systematic review.

Interestingly, our two papers [129, 130] were not identified via our literature search strategies. The main reason such a search is missing is the keywords used in these papers. CDSS was not specified as a keyword in the two papers, although the content was undoubtedly within the scope of this review.

Unfortunately, these two publications may not be the only papers missed by this literature search. This challenge is common to how our current literature databases are organized and how we conduct a literature search. In the future, this challenge can be minimized by carefully choosing all keywords included in the publications to maximize the possibilities found during the literature search. This is also a lesson for all authors.

Conclusions

The reuse, management, and maintenance of CDSS rules are critical and challenging for clinical care. Ontology could facilitate the reusing and sharing of CDSS artifacts, e.g., CDSS rules. However, there is no CDSS ontology yet, and we are building one. We attempt to bridge the high-level visions and operational level efforts via the CDSS ontology to bring tangible applications to the field. Current publications present incomplete technical details on CDSS rules and ontology. We recommend future publications include more detailed information about the architecture diagrams, mechanism of connection among ontology, CDSS rules, EHR/EMR, CDSS rule language, reasoner, the rule engine, validation or authorization of ontology and CDSS rules, the purpose of the ontology, ontology source, and the management and maintenance of CDSS rules. Such details will increase the study’s reproducibility and help readers optimize their designs and development. In addition, we hope the established knowledge framework and summarizations of the included publications can guide future improvements and designs more explicitly.

Data Availability

All data produced in the present study are available upon reasonable request to the authors

Competing interests

None to disclose.

Appendices

Appendix 1: Initial draft of codes and code groups used in reviewing/coding

Appendix 2: Refined version of codes and code groups used in reviewing/coding

Appendix 3: All included paper lists (n = 81) in this systematic literature review

Appendix 4: PRISMA 2020 item checklist reported in our systematic literature review

Appendix 5: PRISMA 2020 for Abstracts checklist reported in our systematic literature review

Acknowledgment

This work is supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award No. R01GM138589 and partially under P20 GM121342. We thank Professor James Cimino for his valuable feedback and suggestions on the manuscript.

Footnotes

  • ↵* Xia Jing, Hua Min, and Yang Gong contribute equally to the manuscript. Therefore they share the first authorship.

References

  1. 1.↵
    Greenes R. Clinical decision support: the road to broad adoption. 2nd ed. San Diego, CA: Elsevier, 2014.
  2. 2.
    Clinical decision support systems: state of the art. AHRQ Publication No. 09-0069-EF; 2009; Rockville, MD.
  3. 3.
    Sittig D, Wright A, Meltzer S, et al. Comparison of clinical knowledge management capabilities of commercially-available and leading internally-developed electronic health records. BMC Med Inform Decis Mak 2011;11
  4. 4.↵
    Sittig D, Wright A, Simonaitis L, et al. The state of the art in clinical knowledge management: An inventory of tools and techniques. Int J Med Inform 2010;79:44–57
    OpenUrlCrossRefPubMedWeb of Science
  5. 5.↵
    Moja L, Polo Friz H, Capobussi M, et al. Effectiveness of a Hospital-Based Computerized Decision Support System on Clinician Recommendations and Patient Outcomes: A Randomized Clinical Trial. JAMA Netw Open 2019;2(12):e1917094 doi: 10.1001/jamanetworkopen.2019.17094[published Online First: Epub Date]|.
    OpenUrlCrossRef
  6. 6.↵
    Enabling Health Care Decisionmaking Through Clinical Decision Support and Knowledge Management. Evidence Report No. 203. (Prepared by the Duke Evidence-based Practice Center under Contract No. 290-2007-10066-I.) AHRQ Publication No. 12-E001-EF.; 2012; Rockville, MD.
  7. 7.
    Kawamoto K, Houlihan C, Balas E, Lobach D. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. J Biomed Inform 2005;330:765
    OpenUrl
  8. 8.↵
    O’Connor P, Sperl-Hillen J, Rush W, et al. Impact of Electronic Health Record Clinical Decision Support on Diabetes Care: A Randomized Trial. Ann Fam Med 2011;9:12–21 doi: 10.1370/afm.1196[published Online First: Epub Date]|.
    OpenUrlAbstract/FREE Full Text
  9. 9.↵
    Middleton B, Sittig DF, Wright A. Clinical Decision Support: a 25 Year Retrospective and a 25 Year Vision. Yearb Med Inform 2016;Suppl 1(Suppl 1):S103–16 doi: 10.15265/IYS-2016-s034[published Online First: Epub Date]|.
    OpenUrlCrossRef
  10. 10.↵
    Wright A, Ash JS, Aaron S, et al. Best practices for preventing malfunctions in rule-based clinical decision support alerts and reminders: Results of a Delphi study. Int J Med Inform 2018;118:78–85 doi: 10.1016/j.ijmedinf.2018.08.001[published Online First: Epub Date]|.
    OpenUrlCrossRef
  11. 11.
    Horsky J, Schiff G, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform 2012;45:1202–16 doi: 10.1016/j.jbi.2012.09.002[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  12. 12.
    Wu R, Peters W, Morgan M. The next generation of clinical decision support: linking evidence to best practice. J Healthc Inf Manag 2002;16:50–55
    OpenUrlPubMed
  13. 13.
    Fox J, Gutenstein M, Khan O, South M, Thomson R. OpenClinical.net: A platform for creating and sharing knowledge and promoting best practice in healthcare. Computers in Industry 2015;66:63–72
    OpenUrl
  14. 14.↵
    Teich J, Osheroff J, Pifer E, Sittig D, Jenders R, Panel CER. Clinical decision support in electronic prescribing: recommendations and an action plan: report of the joint clinical decision support workgroup. J Am Med Inform Assoc 2005;12:365–76 doi: 10.1197/jamia.M1822[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  15. 15.↵
    Sittig D, Wright A, Osheroff J, et al. Grand challenges in clinical decision support. J Biomed Inform 2008;41:387–92
    OpenUrlCrossRefPubMedWeb of Science
  16. 16.↵
    McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med 1976;295(24):1351–5 doi: 10.1056/nejm197612092952405[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMedWeb of Science
  17. 17.↵
    Cimino JJ. An integrated approach to computer-based decision support at the point of care. Transactions of the American Clinical and Climatological Associateion 2007;118:273–88
    OpenUrl
  18. 18.↵
    Jing X, Himawan L, Law T. Availability and usage of clinical decision support systems (CDSSs) in office-based primary care settings in the USA. BMJ Health Care Inform 2019;26(1) doi: 10.1136/bmjhci-2019-100015[published Online First: Epub Date]|.
    OpenUrlAbstract/FREE Full Text
  19. 19.↵
    Borbolla D, Otero C, Lobach DF, et al. Implementation of a clinical decision support system using a service model: results of a feasibility study. Stud Health Technol Inform 2010;160(Pt 2):816–20
    OpenUrlPubMed
  20. 20.↵
    Goldberg HS, Paterno MD, Rocha BH, et al. A highly scalable, interoperable clinical decision support service. J Am Med Inform Assoc 2014;21(e1):e55–62 doi: 10.1136/amiajnl-2013-001990[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  21. 21.↵
    Nguyen BP, Reese T, Decker S, Malone D, Boyce RD, Beyan O. Implementation of Clinical Decision Support Services to Detect Potential Drug-Drug Interaction Using Clinical Quality Language. Stud Health Technol Inform 2019;264:724–28 doi: 10.3233/shti190318[published Online First: Epub Date]|.
    OpenUrlCrossRef
  22. 22.↵
    Linder J, Schnipper J, Volk L, et al. Clinical decision support to improve antibiotic prescribing for acute respiratory infections: results of a pilot study. AMIA Annu Symp Proc, 2007:468–72.
  23. 23.↵
    Ray H, Boxwala A, Anantraman V, Ohno-Machado L. Providing context-sensitive decision-support based on WHO guidelines. Proc AMIA Symp, 2002:637–41.
  24. 24.
    Barth C, Tobman M, Nätscher C, Sussmann H, Horsch A. Fusing a systematic and a case-based repository for medical decision support. Stud Health Technol Inform 2003;95:560–4
    OpenUrlPubMed
  25. 25.↵
    Haug PJ, Ferraro JP, Holmen J, et al. An ontology-driven, diagnostic modeling system. J Am Med Inform Assoc 2013;20(e1):e102–10 doi: 10.1136/amiajnl-2012-001376[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  26. 26.↵
    Del Fiol G, Curtis C, Cimino JJ, et al. Disseminating context-specific access to online knowledge resources within electronic health record systems, 2013.
  27. 27.
    Del Fiol G, Williams MS, Maram N, Rocha RA, Wood GM, Mitchell JA. Integrating genetic information resources with an EHR. AMIA Annu Symp Proc 2006 2006:904
  28. 28.
    Practical choices for infobutton customization: experience from four sites. AMIA … Annual Symposium proceedings / AMIA Symposium. AMIA Symposium; 2013.
  29. 29.
    G DF, PJ H, JJ C, SP N, C N, JA M. Effectiveness of topic-specific infobuttons: a randomized controlled trial. J Am Med Inform Assoc 2008;15:752–59
    OpenUrl
  30. 30.↵
    Cimino JJ. An integrated approach to computer-based decision support at the point of care. Trans Am Clin Climatol Assoc 2007;118:273–88
    OpenUrlPubMed
  31. 31.↵
    Patkar V, Acosta D, Davidson T, Jones A, J F, Keshtgar M. Using computerised decision support to improve compliance of cancer multidisciplinary meetings with evidence-based guidance. BMJ Open 2012;2 doi: http://dx.doi.org/10.1136/bmjopen-2011-000439[published Online First: Epub Date]|.
  32. 32.
    Sutton DR, Fox J. The syntax and semantics of the PROforma guideline modeling language. J Am Med Inform Assoc 2003;10(5):433–43 doi: 10.1197/jamia.M1264[published Online First: Epub Date]|.
    OpenUrlCrossRef
  33. 33.↵
    Del Fiol G, Kohlmann W, Bradshaw RL, et al. Standards-Based Clinical Decision Support Platform to Manage Patients Who Meet Guideline-Based Criteria for Genetic Evaluation of Familial Cancer. JCO Clin Cancer Inform 2020;4:1–9 doi: 10.1200/cci.19.00120[published Online First: Epub Date]|.
    OpenUrlCrossRef
  34. 34.↵
    Hripcsak G. Writing Arden Syntax medical logic modules. Computers in Biology and Medicine 1994;24(5):331–63 doi: doi:10.1016/0010-4825(94)90002-7[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMedWeb of Science
  35. 35.↵
    Zhou L, Karipineni N, Lewis J, et al. A study of diverse clinical decision support rule authoring environments and requirements for integration. BMC Med Inform Decis Mak 2012;12:128 doi: 10.1186/1472-6947-12-128[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  36. 36.↵
    Davies J, Fensel D, Harmelen FV. Towards the semantic web: Ontology-driven knowledge management. Chichester: John Wiley & Sons, 2003.
  37. 37.↵
    Gruber T. What is an ontology? Knowledge Acquisition 1993;5:199–220
    OpenUrlCrossRefWeb of Science
  38. 38.↵
    Musen MA. Dimensions of knowledge sharing and reuse. Computers and Biomedical Research 1992;25:435–67
    OpenUrl
  39. 39.↵
    Whetzel P, Noy N, Shah N, et al. BioPortal: enhanced functionality via new Web services from the National Center for Biomedical Ontology to access and use ontologies in software applications. Nucleic Acids Res 2011;39:W541–45
    OpenUrlCrossRefPubMedWeb of Science
  40. 40.↵
    Dissanayake PI, Colicchio TK, Cimino JJ. Using clinical reasoning ontologies to make smarter clinical decision support systems: a systematic review and data synthesis. J Am Med Inform Assoc 2020;27(1):159–74 doi: 10.1093/jamia/ocz169[published Online First: Epub Date]|.
    OpenUrlCrossRef
  41. 41.↵
    Jing X, Min H, Gong Y, et al. A clinical decision support system (CDSS) ontology to facilitate portable vaccination CDSS rules: preliminary results. AMIA 2021. San Diego, CA., 2021.
  42. 42.↵
    Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. J Clin Epidemiol 2021;134:178–89 doi: 10.1016/j.jclinepi.2021.03.001[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  43. 43.↵
    Manjarrés Riesco A, Martínez Tomás R, Mira Mira J. A customisable framework for the assessment of therapies in the solution of therapy decision tasks. Artif Intell Med 2000;18(1):57–82 doi: 10.1016/s0933-3657(99)00029-9[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  44. 44.
    Abidi SS, Manickam S. Transforming XML-based electronic patient records for use in medical case based reasoning systems. Stud Health Technol Inform 2000;77:709–13
    OpenUrlPubMedWeb of Science
  45. 45.
    De Clercq PA, Blom JA, Hasman A, Korsten HH. GASTON: an architecture for the acquisition and execution of clinical guideline-application tasks. Med Inform Internet Med 2000;25(4):247–63 doi: 10.1080/146392300455558[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMedWeb of Science
  46. 46.
    Payne TH, Savarino J, Marshall R, Hoey CT. Use of a clinical event monitor to prevent and detect medication errors. Proc AMIA Symp 2000:640–4
  47. 47.↵
    Achour SL, Dojat M, Rieux C, Bierling P, Lepage E. A UMLS-based knowledge acquisition tool for rule-based clinical decision support system development. J Am Med Inform Assoc 2001;8(4):351–60 doi: 10.1136/jamia.2001.0080351[published Online First: Epub Date]|.
    OpenUrlCrossRef
  48. 48.
    Séroussi B, Bouaud J, Dréau H, et al. ASTI: a guideline-based drug-ordering system for primary care. Stud Health Technol Inform 2001;84(Pt 1):528–32
    OpenUrlPubMed
  49. 49.↵
    Karadimas HC, Chailloleau C, Hemery F, Simonnet J, Lepage E. Arden/J: an architecture for MLM execution on the Java platform. J Am Med Inform Assoc 2002;9(4):359–68 doi: 10.1197/jamia.m0985[published Online First: Epub Date]|.
    OpenUrlCrossRef
  50. 50.
    Das AK, Musen MA. SYNCHRONUS: a reusable software module for temporal integration. Proc AMIA Symp 2002:195–9
  51. 51.
    Poon EG, Wang SJ, Gandhi TK, Bates DW, Kuperman GJ. Design and implementation of a comprehensive outpatient Results Manager. J Biomed Inform 2003;36(1-2):80–91 doi: 10.1016/s1532-0464(03)00061-3[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMedWeb of Science
  52. 52.
    Liaw ST, Sulaiman N, Pearce C, et al. Falls prevention within the Australian general practice data model: methodology, information model, and terminology issues. J Am Med Inform Assoc 2003;10(5):425–32 doi: 10.1197/jamia.M1281[published Online First: Epub Date]|.
    OpenUrlCrossRef
  53. 53.
    Greenes RA, Sordo M, Zaccagnini D, Meyer M, Kuperman GJ. Design of a standards-based external rules engine for decision support in a variety of application contexts: report of a feasibility study at Partners HealthCare System. Stud Health Technol Inform 2004;107(Pt 1):611–5
    OpenUrlPubMed
  54. 54.
    Ebrahiminia V, Riou C, Seroussi B, et al. Design of a decision support system for chronic diseases coupling generic therapeutic algorithms with guideline-based specific rules. Stud Health Technol Inform 2006;124:483–8
    OpenUrlPubMed
  55. 55.↵
    Kashyap V, Morales A, Hongsermeier T. On implementing clinical decision support: achieving scalability and maintainability by combining business rules and ontologies. AMIA Annu Symp Proc 2006;2006:414–8
    OpenUrl
  56. 56.
    Verlaene K, Joosen W, Verbaeten P. Arriclides: An Architecture Integrating Clinical Decision Support Models. Proceedings of the 40th Annual Hawaii International Conference on System Sciences: IEEE Computer Society, 2007:135.
  57. 57.↵
    Abidi SR. Ontology-Based Modeling of Breast Cancer Follow-up Clinical Practice Guideline for Providing Clinical Decision Support. Proceedings of the Twentieth IEEE International Symposium on Computer-Based Medical Systems: IEEE Computer Society, 2007:542–47.
  58. 58.
    Jannin P, Morandi X. Surgical models for computer-assisted neurosurgery. Neuroimage 2007;37(3):783–91 doi: 10.1016/j.neuroimage.2007.05.034[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMedWeb of Science
  59. 59.↵
    Stacey M, McGregor C, Tracy M. An architecture for multi-dimensional temporal abstraction and its application to support neonatal intensive care. Annu Int Conf IEEE Eng Med Biol Soc 2007;2007:3752–6 doi: 10.1109/iembs.2007.4353148[published Online First: Epub Date]|.
    OpenUrlCrossRef
  60. 60.
    Papageorgiou E, Stylios C, Groumpos P. Novel Architecture for supporting medical decision making of different data types based on Fuzzy Cognitive Map Framework. Annu Int Conf IEEE Eng Med Biol Soc 2007;2007:1192–5 doi: 10.1109/iembs.2007.4352510[published Online First: Epub Date]|.
    OpenUrlCrossRef
  61. 61.
    Cornalba C, Bellazzi RG, Bellazzi R. Building a normative decision support system for clinical and operational risk management in hemodialysis. IEEE Trans Inf Technol Biomed 2008;12(5):678–86 doi: 10.1109/titb.2008.920781[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  62. 62.
    Farion K, Michalowski W, Wilk S, O’Sullivan D, Rubin S, Weiss D. Clinical decision support system for point of care use--ontology-driven design and software implementation. Methods Inf Med 2009;48(4):381–90 doi: 10.3414/me0574[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  63. 63.
    Dao TT, Marin F, Ho Ba Tho MC. Clinical validated Computer-Aided Decision System to the clubfeet deformities. Annu Int Conf IEEE Eng Med Biol Soc 2009;2009:6230–3 doi: 10.1109/iembs.2009.5334650[published Online First: Epub Date]|.
    OpenUrlCrossRef
  64. 64.↵
    Zhou Q. A Clinical Decision Support System for Metabolism Synthesis. Proceedings of the 2009 International Conference on Computational Intelligence and Natural Computing - Volume 02: IEEE Computer Society, 2009:323–25.
  65. 65.↵
    Carenini M. ReMINE: an ontology-based risk management platform. Stud Health Technol Inform 2009;148:32–42
    OpenUrlPubMed
  66. 66.
    Lee J, Kim J, Cho I, Kim Y. Integration of workflow and rule engines for clinical decision support services. Stud Health Technol Inform 2010;160(Pt 2):811–5
    OpenUrlPubMed
  67. 67.
    Ongenae F, Dhaene T, Turck FD, Benoit D, Decruyenaere J. Design of a probabilistic ontology-based clinical decision support system for classifying temporal patterns in the ICU: A sepsis case study. Proceedings of the 2010 IEEE 23rd International Symposium on Computer-Based Medical Systems: IEEE Computer Society, 2010:389–94.
  68. 68.
    Basilakis J, Lovell N, Redmond S, Celler B. Design of a decision-support architecture for management of remotely monitored patients. IEEE Transactions on Information Technology in Biomedicine 2010;14:1216–26
    OpenUrlPubMed
  69. 69.
    Wilk S, Michalowski W, Farion K, Sayyad Shirabad J. MET3-AE system to support management of pediatric asthma exacerbation in the emergency department. Stud Health Technol Inform 2010;160(Pt 2):841–5
    OpenUrlPubMed
  70. 70.
    Bouamrane M-M, Rector A, Hurrell M. Using OWL ontologies for adaptive patient information modelling and preoperative clinical decision support. Knowl. Inf. Syst. 2011;29(2):405–18 doi: 10.1007/s10115-010-0351-7[published Online First: Epub Date]|.
    OpenUrlCrossRef
  71. 71.
    Dao TT, Marin F, Bensahel H, Ho Ba Tho MC. Computer-aided decision system for the clubfeet deformities. Adv Exp Med Biol 2011;696:623–35 doi: 10.1007/978-1-4419-7046-6_64[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  72. 72.
    Cao F, Sun X, Wang X, Li B, Li J, Pan Y. Ontology-based knowledge management for personalized adverse drug events detection. Stud Health Technol Inform 2011;169:699–703
    OpenUrlPubMed
  73. 73.
    Lee CS, Wang MH. A fuzzy expert system for diabetes decision support application. IEEE Trans Syst Man Cybern B Cybern 2011;41(1):139–53 doi: 10.1109/tsmcb.2010.2048899[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  74. 74.
    Riaño D, Real F, López-Vallverdú JA, et al. An ontology-based personalization of health-care knowledge to support clinical decisions for chronically ill patients. J Biomed Inform 2012;45(3):429–46 doi: 10.1016/j.jbi.2011.12.008[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  75. 75.
    Bright TJ, Yoko Furuya E, Kuperman GJ, Cimino JJ, Bakken S. Development and evaluation of an ontology for guiding appropriate antibiotic prescribing. J Biomed Inform 2012;45(1):120–8 doi: 10.1016/j.jbi.2011.10.001[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  76. 76.
    Koutkias V, Kilintzis V, Stalidis G, et al. Knowledge engineering for adverse drug event prevention: on the design and development of a uniform, contextualized and sustainable knowledge-based framework. J Biomed Inform 2012;45(3):495–506 doi: 10.1016/j.jbi.2012.01.007[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  77. 77.
    Grando A, Farrish S, Boyd C, Boxwala A. Ontological approach for safe and effective polypharmacy prescription. AMIA Annu Symp Proc 2012;2012:291–300
    OpenUrl
  78. 78.
    Chniti A, Boussadi A, Degoulet P, Albert P, Charlet J. Pharmaceutical validation of medication orders using an OWL Ontology and Business Rules. Stud Health Technol Inform 2012;180:1224–6
    OpenUrlPubMed
  79. 79.
    Paterno MD, Goldberg HS, Simonaitis L, et al. Using a service oriented architecture approach to clinical decision support: performance results from two CDS Consortium demonstrations. AMIA Annu Symp Proc 2012;2012:690–8
    OpenUrl
  80. 80.
    Yao W, Kumar A. CONFlexFlow: Integrating Flexible clinical pathways into clinical decision support systems using context and rules. Decis. Support Syst. 2013;55(2):499–515 doi: 10.1016/j.dss.2012.10.008[published Online First: Epub Date]|.
    OpenUrlCrossRef
  81. 81.
    Artetxe A, Sanchez E, Toro C, et al. IMPACT OF REFLEXIVE ONTOLOGIES IN SEMANTIC CLINICAL DECISION SUPPORT SYSTEMS. Cybern. Syst. 2013;44(2–3):187–203 doi: 10.1080/01969722.2013.762256[published Online First: Epub Date]|.
    OpenUrlCrossRef
  82. 82.↵
    Farkash A, Timm JT, Waks Z. A model-driven approach to clinical practice guidelines representation and evaluation using standards. Stud Health Technol Inform 2013;192:200–4
    OpenUrl
  83. 83.↵
    SáEz C, Bresó A, Vicente J, Robles M, GarcíA-GóMez JM. An HL7-CDA wrapper for facilitating semantic interoperability to rule-based Clinical Decision Support Systems. Comput. Methods Prog. Biomed. 2013;109(3):239–49 doi: 10.1016/j.cmpb.2012.10.003[published Online First: Epub Date]|.
    OpenUrlCrossRef
  84. 84.
    Corrigan D, Taweel A, Fahey T, Arvanitis T, Delaney B. An ontological treatment of clinical prediction rules implementing the Alvarado score. Stud Health Technol Inform 2013;186:103–7
    OpenUrl
  85. 85.
    Shojanoori R, Juric R. Semantic remote patient monitoring system. Telemed J E Health 2013;19(2):129–36 doi: 10.1089/tmj.2012.0128[published Online First: Epub Date]|.
    OpenUrlCrossRef
  86. 86.
    Wilk S, Michalowski W, O’Sullivan D, et al. A task-based support architecture for developing point-of-care clinical decision support systems for the emergency department. Methods Inf Med 2013;52(1):18–32 doi: 10.3414/me11-01-0099[published Online First: Epub Date]|.
    OpenUrlCrossRef
  87. 87.
    Yılmaz Ö, Erdur RC, Türksever M. SAMS--a systems architecture for developing intelligent health information systems. J Med Syst 2013;37(6):9989 doi: 10.1007/s10916-013-9989-5[published Online First: Epub Date]|.
    OpenUrlCrossRef
  88. 88.↵
    Bau CT, Chen RC, Huang CY. Construction of a clinical decision support system for undergoing surgery based on domain ontology and rules reasoning. Telemed J E Health 2014;20(5):460–72 doi: 10.1089/tmj.2013.0221[published Online First: Epub Date]|.
    OpenUrlCrossRef
  89. 89.
    Wang HQ, Zhou TS, Tian LL, Qian YM, Li JS. Creating hospital-specific customized clinical pathways by applying semantic reasoning to clinical data. J Biomed Inform 2014;52:354–63 doi: 10.1016/j.jbi.2014.07.017[published Online First: Epub Date]|.
    OpenUrlCrossRef
  90. 90.↵
    Sesen MB, Peake MD, Banares-Alcantara R, et al. Lung Cancer Assistant: a hybrid clinical decision support application for lung cancer care. J R Soc Interface 2014;11(98):20140534 doi: 10.1098/rsif.2014.0534[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  91. 91.
    Gallerani M, Pelizzola D, Pivanti M, Boni M, Lamma E, Bellodi E. Appropriateness of Repeated Execution of Laboratory Examinations: A CDSS Approach. Proceedings of the 2014 IEEE International Conference on Healthcare Informatics: IEEE Computer Society, 2014:161–69.
  92. 92.
    Stewart SA, Abidi S, Parker L, Bernstein M, Abidi SS. Clinical guideline-driven personalized self-management diary for paediatric cancer survivors. Stud Health Technol Inform 2014;205:18–22
    OpenUrl
  93. 93.
    Shen Y, Colloc J, Jacquet-Andrieu A, Lei K. Emerging medical informatics with case-based reasoning for aiding clinical decision in multi-agent system. J Biomed Inform 2015;56:307–17 doi: 10.1016/j.jbi.2015.06.012[published Online First: Epub Date]|.
    OpenUrlCrossRef
  94. 94.↵
    Delaney BC, Curcin V, Andreasson A, et al. Translational Medicine and Patient Safety in Europe: TRANSFoRm--Architecture for the Learning Health System in Europe. Biomed Res Int 2015;2015:961526 doi: 10.1155/2015/961526[published Online First: Epub Date]|.
    OpenUrlCrossRef
  95. 95.
    Jafarpour B, Abidi SR, Ahmad AM, Abidi SS. INITIATE: An Intelligent Adaptive Alert Environment. Stud Health Technol Inform 2015;216:285–9
    OpenUrl
  96. 96.
    Robles-Bykbaeva V, López-Noresb M, Pazos-Ariasb J, Quisi-Peraltaa D, García-Duqueb J. An Ecosystem of Intelligent ICT Tools for Speech-Language Therapy Based on a Formal Knowledge Model. MEDINFO 2015, 2015:50–54.
    OpenUrl
  97. 97.↵
    Wilk S, Kezadri-Hamiaz M, Rosu D, et al. Using Semantic Components to Represent Dynamics of an Interdisciplinary Healthcare Team in a Multi-Agent Decision Support System. J Med Syst 2016;40(2):42 doi: 10.1007/s10916-015-0375-3[published Online First: Epub Date]|.
    OpenUrlCrossRef
  98. 98.
    Abidi SR, Cox J, Abusharekh A, Hashemian N, Abidi SS. A Digital Health System to Assist Family Physicians to Safely Prescribe NOAC Medications. Stud Health Technol Inform 2016;228:519–23
    OpenUrl
  99. 99.
    Zhang Y-F, Tian Y, Zhou T-S, Araki K, Li J-S. Integrating HL7 RIM and ontology for unified knowledge and data representation in clinical decision support systems. Comput. Methods Prog. Biomed. 2016;123(C):94–108 doi: 10.1016/j.cmpb.2015.09.020[published Online First: Epub Date]|.
    OpenUrlCrossRef
  100. 100.
    Goldberg HS, Paterno MD, Grundmeier RW, et al. Use of a remote clinical decision support service for a multicenter trial to implement prediction rules for children with minor blunt head trauma. Int J Med Inform 2016;87:101–10 doi: 10.1016/j.ijmedinf.2015.12.002[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  101. 101.↵
    Marco-Ruiz L, Pedrinaci C, Maldonado JA, Panziera L, Chen R, Bellika JG. Publication, discovery and interoperability of Clinical Decision Support Systems: A Linked Data approach. J Biomed Inform 2016;62:243–64 doi: 10.1016/j.jbi.2016.07.011[published Online First: Epub Date]|.
    OpenUrlCrossRef
  102. 102.
    Zhang YF, Gou L, Zhou TS, et al. An ontology-based approach to patient follow-up assessment for continuous and personalized chronic disease management. J Biomed Inform 2017;72:45–59 doi: 10.1016/j.jbi.2017.06.021[published Online First: Epub Date]|.
    OpenUrlCrossRef
  103. 103.
    Shang Y, Wang Y, Gou L, Wu C, Zhou T, Li JS. Development of a Service-Oriented Sharable Clinical Decision Support System Based on Ontology for Chronic Disease. Stud Health Technol Inform 2017;245:1153–57
    OpenUrl
  104. 104.
    Chen RC, Jiang HQ, Huang CY, Bau CT. Clinical Decision Support System for Diabetes Based on Ontology Reasoning and TOPSIS Analysis. J Healthc Eng 2017;2017:4307508 doi: 10.1155/2017/4307508[published Online First: Epub Date]|.
    OpenUrlCrossRef
  105. 105.↵
    Kopanitsa G. Integration of Hospital Information and Clinical Decision Support Systems to Enable the Reuse of Electronic Health Record Data. Methods Inf Med 2017;56(3):238–47 doi: 10.3414/me16-01-0057[published Online First: Epub Date]|.
    OpenUrlCrossRef
  106. 106.↵
    Abidi S. A Knowledge-Modeling Approach to Integrate Multiple Clinical Practice Guidelines to Provide Evidence-Based Clinical Decision Support for Managing Comorbid Conditions. J Med Syst 2017;41(12):193 doi: 10.1007/s10916-017-0841-1[published Online First: Epub Date]|.
    OpenUrlCrossRef
  107. 107.
    Shen Y, Yuan K, Chen D, et al. An ontology-driven clinical decision support system (IDDAP) for infectious disease diagnosis and antibiotic prescription. Artif Intell Med 2018;86:20–32 doi: 10.1016/j.artmed.2018.01.003[published Online First: Epub Date]|.
    OpenUrlCrossRef
  108. 108.
    Nakawala H, Ferrigno G, De Momi E. Development of an intelligent surgical training system for Thoracentesis. Artif Intell Med 2018;84:50–63 doi: 10.1016/j.artmed.2017.10.004[published Online First: Epub Date]|.
    OpenUrlCrossRef
  109. 109.↵
    Séroussi B, Guézennec G, Lamy JB, et al. Reconciliation of multiple guidelines for decision support: a case study on the multidisciplinary management of breast cancer within the DESIREE project. AMIA Annu Symp Proc 2018:1527–36
  110. 110.↵
    Winter A, Stäubert S, Ammon D, et al. Smart Medical Information Technology for Healthcare (SMITH). Methods Inf Med 2018;57(S 01):e92–e105 doi: 10.3414/me18-02-0004[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  111. 111.↵
    Seroussi B, Lamy JB, Muro N, et al. Implementing Guideline-Based, Experience-Based, and Case-Based Approaches to Enrich Decision Support for the Management of Breast Cancer Patients in the DESIREE Project. Stud Health Technol Inform 2018;255:190–94
    OpenUrl
  112. 112.
    Jin W, Kim DH. Design and Implementation of e-Health System Based on Semantic Sensor Network Using IETF YANG. Sensors (Basel) 2018;18(2) doi: 10.3390/s18020629[published Online First: Epub Date]|.
    OpenUrlCrossRef
  113. 113.
    Román-Villarán E, Pérez-Leon FP, Escobar-Rodriguez GA, Martínez-García A, Álvarez-Romero C, Parra-Calderón CL. An Ontology-Based Personalized Decision Support System for Use in the Complex Chronically Ill Patient. Stud Health Technol Inform 2019;264:758–62 doi: 10.3233/shti190325[published Online First: Epub Date]|.
    OpenUrlCrossRef
  114. 114.
    Semenov I, Osenev R, Gerasimov S, Kopanitsa G, Denisov D, Andreychuk Y. Experience in Developing an FHIR Medical Data Management Platform to Provide Clinical Decision Support. Int J Environ Res Public Health 2019;17(1) doi: 10.3390/ijerph17010073[published Online First: Epub Date]|.
    OpenUrlCrossRef
  115. 115.↵
    Jafarpour B, Raza Abidi S, Van Woensel W, Raza Abidi SS. Execution-time integration of clinical practice guidelines to provide decision support for comorbid conditions. Artif Intell Med 2019;94:117–37 doi: 10.1016/j.artmed.2019.02.003[published Online First: Epub Date]|.
    OpenUrlCrossRef
  116. 116.
    El-Sappagh S, Ali F, Hendawi A, Jang JH, Kwak KS. A mobile health monitoring-and-treatment system based on integration of the SSN sensor ontology and the HL7 FHIR standard. BMC Med Inform Decis Mak 2019;19(1):97 doi: 10.1186/s12911-019-0806-z[published Online First: Epub Date]|.
    OpenUrlCrossRef
  117. 117.↵
    Maldonado JA, Marcos M, Fernández-Breis JT, Giménez-Solano VM, Legaz-García MDC, Martínez-Salvador B. CLIN-IK-LINKS: A platform for the design and execution of clinical data transformation and reasoning workflows. Comput Methods Programs Biomed 2020;197:105616 doi: 10.1016/j.cmpb.2020.105616[published Online First: Epub Date]|.
    OpenUrlCrossRef
  118. 118.↵
    Regier R, Gurjar R, Rocha RA. A clinical rule editor in an electronic medical record setting: development, design, and implementation. AMIA Annu Symp Proc 2009;2009:537–41
    OpenUrlPubMed
  119. 119.↵
    Sordo M, Rocha BH, Morales AA, et al. Modeling decision support rule interactions in a clinical setting. Stud Health Technol Inform 2013;192:908–12
    OpenUrl
  120. 120.↵
    Bizer C, Heath T, Berners-Lee T. Linked Data - The Story So Far. Int. J. Semantic Web Inf. Syst. 2009;5:1–22
    OpenUrl
  121. 121.↵
    Sandia National Laboratories. Jess, the Rule Engine for the Java Platform. Secondary Jess, the Rule Engine for the Java Platform Nov 25th, 2013. https://jess.sandia.gov/.
  122. 122.↵
    The Apache Software Foundation. Apache Jena. Secondary Apache Jena. https://jena.apache.org/.
  123. 123.↵
    Red Hat. Drools. Secondary Drools. https://www.drools.org/.
  124. 124.↵
    Parsia B, Matentzoglu N, Gonçalves RS, Glimm B, Steigmiller A. The OWL Reasoner Evaluation (ORE) 2015 Competition Report. J Autom Reason 2017;59(4):455–82 doi: 10.1007/s10817-017-9406-8[published Online First: Epub Date]|.
    OpenUrlCrossRef
  125. 125.↵
    Sirin E, Parsia B, Grau BC, Kalyanpur A, Katz Y. Pellet: A practical OWL-DL reasoner. Journal of Web Semantics 2007;5(2):51–53 doi: https://doi.org/10.1016/j.websem.2007.03.004[published Online First: Epub Date]|.
    OpenUrl
  126. 126.↵
    Supporting Rule System Interoperability on the Semantic Web with SWRL; 2005; Berlin, Heidelberg. Springer Berlin Heidelberg.
  127. 127.↵
    Musen M, Team tP. The Protégé Project: A Look Back and a Look Forward. AI Matters 2015;1:4–12 doi: 10.1145/2757001.2757003[published Online First: Epub Date]|.
    OpenUrlCrossRefPubMed
  128. 128.↵
    Farooq K. A Novel Ontology and Machine Learning Driven Hybrid Clinical Decision Support Framework for Cardiovascular Preventative Care. GRIN Verlag, 2016.
  129. 129.↵
    Jing X, Kay S, Marley T, Hardiker NR, Cimino JJ. Incorporating personalized gene sequence variants, molecular genetics knowledge, and health knowledge into an EHR prototype based on the Continuity of Care Record Standard. J Biomed Inform 2012;45:82–92
    OpenUrlCrossRefPubMed
  130. 130.↵
    Jing X, Kay S, Marley T, Hardiker NR. Integration of an OWL-DL knowledge base with an EHR prototype and providing customized information. J Med Syst 2014;38
Back to top
Posted May 16, 2022.
Download PDF

Supplementary Material

Data/Code
Email
A systematic review of ontology-based clinical decision support system rules: usage, management, and interoperability
Xia Jing, Hua Min, Yang Gong, Dean F. Sittig, Paul Biondich, David Robinson, Tim Law, Adam Wright, Christian Nøhr, Arild Faxvaag, Lior Rennert, Nina Hubig, Ronald Gimbel
medRxiv 2022.05.11.22274984; doi: https://doi.org/10.1101/2022.05.11.22274984
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools

Subject Area