This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on https://medinform.jmir.org/, as well as this copyright and license information must be included.
Clinical decision support systems (CDSSs) are important for the quality and safety of health care delivery. Although CDSS rules guide CDSS behavior, they are not routinely shared and reused.
Ontologies have the potential to promote the reuse of CDSS rules. Therefore, we systematically screened the literature to elaborate on the current status of ontologies applied in CDSS rules, such as rule management, which uses captured CDSS rule usage data and user feedback data to tailor CDSS services to be more accurate, and maintenance, which updates CDSS rules. Through this systematic literature review, we aim to identify the frontiers of ontologies used in CDSS rules.
The literature search was focused on the intersection of ontologies; clinical decision support; and rules in PubMed, the Association for Computing Machinery (ACM) Digital Library, and the Nursing & Allied Health Database. Grounded theory and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines were followed. One author initiated the screening and literature review, while 2 authors validated the processes and results independently. The inclusion and exclusion criteria were developed and refined iteratively.
CDSSs were primarily used to manage chronic conditions, alerts for medication prescriptions, reminders for immunizations and preventive services, diagnoses, and treatment recommendations among 81 included publications. The CDSS rules were presented in Semantic Web Rule Language, Jess, or Jena formats. Despite the fact that ontologies have been used to provide medical knowledge, CDSS rules, and terminologies, they have not been used in CDSS rule management or to facilitate the reuse of CDSS rules.
Ontologies have been used to organize and represent medical knowledge, controlled vocabularies, and the content of CDSS rules. So far, there has been little reuse of CDSS rules. More work is needed to improve the reusability and interoperability of CDSS rules. This review identified and described the ontologies that, despite their limitations, enable Semantic Web technologies and their applications in CDSS rules.
For more than half a century, clinical decision support systems (CDSSs) have been developed and used in clinical care delivery [
CDSS rules, which function similarly to the human central nervous system, direct the behaviors of a CDSS during operations by incorporating patient data, contextual information, and medical domain knowledge. The central role of CDSS rules is a decisive factor in the relevance and usefulness of a CDSS in the overall clinical workflow, which impacts whether a CDSS is adopted and routinely used. CDSS rules can be written in Arden syntax [
Ontologies have been successfully applied to generate and supply domain knowledge in the use, reuse, sharing, and interoperability of information. Ontologies are seen as promising solutions to the challenges of managing and maintaining CDSS rules across institutional boundaries. The Semantic Web is a technology enabled by ontology [
Reusing and sharing CDSS rules are important, but they are not yet routine operations; thus, we conducted this systematic literature review. This study aims to expand on the current state of using ontologies in CDSS rules by conducting a systematic review of the literature on the intersection of CDSS rules, Semantic Web technologies (particularly ontologies), and use of ontologies in CDSSs. The review is expected to provide a comprehensive view of using ontologies in CDSS rules, with granular details. The results could serve as a basis to form a knowledge framework of the topic that may inspire future research. The research question we intend to answer with this systematic literature review is as follows: What is the current state of using semantic technologies, particularly ontologies, to leverage CDSS rule interoperability? Furthermore, the manually annotated results of selected publications could serve as gold standards for automatically identifying relevant entities in the literature.
For PubMed, the following search was conducted: (clinical decision support systems[MeSH Terms]) AND (ontolog*[Title/abstract] OR rule*[Title/abstract]). For ACM Digital Library, the following search was conducted within the scope of the ACM Guide to Computing Literature: [[Publication Title: “clinical decision support*”] OR [Publication Title: cds*]] AND [[Publication Title: ontolog*] OR [Abstract: ontolog*] OR [Publication Title: rule*] OR [Abstract: rule*]]. For the NAHD, the following search was limited to peer-reviewed publications: mesh(clinical decision support) AND (ti(ontology) OR ti(ontologies) OR ab(ontology) OR ab(ontologies) OR ti(rule) OR ti(rules) OR ab(rule) OR ab(rules)).
General workflow of the systematic literature review.
The inclusion criteria were as follows: text was written in English; full-text publication was available; ontologies were designed to be implemented or were already implemented in CDSSs, particularly related to CDSS rules; content included the granularity of CDSS rules; ontologies were designed to be integrated or were already integrated with health information systems (eg, EHRs), either in a production system or a prototype, with at least one architecture diagram, applied in clinical domains or designed for clinical domains to support health care providers; the publication was peer-reviewed; and details on the integration of CDSSs and EHRs were present for evaluation studies.
The exclusion criteria were as follows: only CDSS rules were included, regardless of the stage of the CDSS rule lifecycle (ie, development, identification, refinement, validation, evaluation, or implementation) or there was no mention of integration or ontologies; only ontologies were developed, evaluated, and validated, or there was no mention of integration or a CDSS; the system was designed without mentioning the granularity of CDSS rules or ontologies; and nonclinical decisions, such as administrative or management decisions (eg, supply chain management), were described.
The first 100 papers were screened by all 3 authors (XJ, HM, and YG) independently. The first 100 retrieved papers were initially screened by 1 author (XJ) to draft initial inclusion and exclusion criteria. The inclusion and exclusion criteria were refined and adjusted by 2 authors (HM and YG) during the iterative screening, review, analysis, and discussions. Further, 2 authors (HM and YG) replicated the screening, and all 3 authors discussed and validated the results. The rest of the papers were then screened by at least 2 authors (XJ and HM, or XJ and YG) independently to determine inclusion. Disagreements were discussed and resolved via iterative rounds of group meetings.
The screening and manual review processes were conducted independently and approved by at least 2 authors. The literature was first screened based on titles, abstracts, and full-text publications when needed. The papers that were included were then manually coded to provide more content analysis and synthesized evidence. The final results were shared among all the authors. All disagreements were settled through group discussions.
We followed grounded theory during the reviewing and manual coding of the included publications. One author (XJ) randomly selected 10 papers from the included 81 papers to start the coding (annotating) based on the focus of this literature review. ATLAS.ti 9 (desktop and web versions; ATLAS.ti Scientific Software Development GmbH), a qualitative data analytic tool, was used for coding. The coding results were discussed by 3 authors (XJ, HM, and YG). The discussion results formed the first draft of codes and code groups (
After coding, the literature was analyzed and synthesized with a focus on several aspects, including CDSS application domains, CDSS mechanisms used in clinical settings, CDSS rule formats, authoring, management, and the roles of ontologies. The 3 authors worked together in an iterative process of analysis and synthetization. After obtaining consensus among all 3 authors, the results were then shared and discussed among all authors. Any concerns, confusions, or disagreements among the authors were resolved through iterative discussions. We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 checklist [
By January 5, 2022, literature searches retrieved 1235 publications from 3 sources. After removing duplicates and examining according to the inclusion and exclusion criteria, 81 publications (
Flowchart of the literature search, screening, and selection. ACM: Association for Computing Machinery.
Initial knowledge framework on clinical decision support systems (CDSSs), CDSS rules, and ontology applications in CDSSs. ICD-10: International Statistical Classification of Diseases and Related Health Problems, 10th Revision; LOINC: Logical Observation Identifiers Names and Codes; MLM: medical logic module; OWL: Web Ontology Language; SNOMED CT: Systematized Nomenclature of Medicine-Clinical Terms; SWRL: Semantic Web Rule Language; UMLS: Unified Medical Language System; XML: Extensive Markup Language.
The majority of the publications (73/81, 90%) included in the review were from PubMed, a dominant source. After removing duplicates, the ACM Library added 8 new publications. After cleaning, discussion, and consolidation, 30 code groups and 221 final codes were used in ATLAS.ti (
PRISMA 2020 is designed to guide the reporting of outcome-oriented studies. Our systematic literature review focused on the design, development, and implementation of CDSSs, particularly related to CDSS rules and ontologies. Therefore, effect measures or certainty assessments were irrelevant items. We reported 19 categorical items (out of 27 categorical items, 26 items out of 42 items;
Over one-third (29/81, 36%) of CDSSs were designed and used for chronic condition management, prediction, or risk assessment, including but not limited to type 1 and 2 diabetes, hypertension, and asthma. Medication prescriptions (13/81, 16%), such as medication ordering, detection of adverse drug events, drug-drug interactions, and cancer care (8/81, 10%), were also significant application domains.
Most CDSS rules were written in Web Ontology Language (OWL; 11/81, 14%), Extensive Markup Language (XML; 10/81, 12%), SWRL (9/81, 11%), Jena rules (5/81, 6%), and medical logic module (MLM; 3/81, 4%). Moreover, 2 publications [
The most significant CDSS rule source is from clinical practice guidelines (36/81, 44%). Other sources of CDSS rules included domain expert input, publications (eg, textbooks and papers), multimedia sources, and internet resources. Data mining results were involved in CDSS rule sources [
There was a lack of technical details regarding rule engines, among which Jena (6/81, 7%), inference engine (6/81, 7%), Jess (4/81, 5%), JBoss (3/81, 4%), guideline engine (3/81, 4%), Drools (2/81, 3%), and Bayes (2/81, 3%) were frequently mentioned.
The majority of the publications did not appear to be focused on interoperability. Few papers that discussed interoperability (
Furthermore, some publications lacked necessary information for explaining the mechanisms of the systems, which can be critical barriers to reproducibility. Some publications lacked critical information, such as CDSS architecture diagrams; CDSS rule engines; CDSS rule languages; backend management methods for CDSS rules; and integration mechanisms among CDSS rules, ontologies, and EHR, EMR, or HIS systems.
In the included publications, ontologies were primarily used as knowledge sources for CDSSs (32/81, 40%) to facilitate classification (7/81, 9%), reasoning, and inference (6/81, 7%; eg, identification recommendations or relationships). Moreover, ontologies were used to specify CDSS rules (12/81, 15%) or to provide general knowledge for the EMR or EHR systems. These 2 applications overlapped in some cases (19/81, 24%; ie, the ontologies were used to provide specified CDSS rules and general knowledge).
In the included publications, the terms “reasoner” and “rule engines” were used interchangeably.
The content and code systems used to represent the content should be included as ontology sources. The content could come from a popular textbook or a clinical practice guideline. The content can be coded in a specific code system, such as SNOMED CT (Systematized Nomenclature of Medicine-Clinical Terms).
Incomplete information for ontology validation is a common issue shown in the literature. Approximately 20 publications mentioned some validation, including validation or evaluation by domain experts (20/81, 25%). Some ontologies were authored by domain experts [
Although ontologies contribute to the content of CDSS rules and have the potential to facilitate interoperable CDSS rules, our systematic review showed that reusing and sharing of CDSS rules have not been achieved. CDSSs have a wide range of clinical application domains, primarily for health care providers, such as chronic condition management, medication ordering, and cancer care. CDSS rules are primarily based on clinical practice guidelines.
Although reusing and sharing CDSS artifacts are well-recognized challenges [
Ontologies were not at the center of any early examples of CDSSs [
Over the years, CDSSs have been successfully applied in clinical care. Unfortunately, CDSS rules are not yet portable. Making CDSS rules more portable is therefore significant work that could be leveraged by ontologies, and our systematic literature review brings us one step closer to that goal. Marco-Ruiz et al also conducted a very relevant systematic literature review. However, their focus was on the interoperability mechanisms used in CDSSs [
Rule engines, which execute rules, patient data, and context information to produce a result, such as an alert or a recommendation, are critical components of CDSSs [
Reasoning via a reasoner is a critical characteristic of many ontologies, even though the current reasoning is still in first-degree logic. Reasoning can be used for the following 3 main functions: consistency check, classification, and realization [
This review has demonstrated unique insights about CDSS rules, ontologies, and ontology applications, particularly in CDSS rule management and maintenance, and has presented several distinct characteristics that complement the existing literature. An earlier review [
Our systematic review demonstrated the state-of-the-art applications of ontologies in CDSS rules. These applications have a lot of potential for reusing and sharing CDSS artifacts. However, none of the existing papers elaborated or demonstrated how ontologies enable portable CDSS rules. Although some authors recognized this benefit [
Inconsistent or missing information about CDSS rule languages, CDSS rule engines, and CDSS evaluation details was identified. In CDSS evaluation, there was commonly no information about how the evaluation was conducted or who performed the evaluation. There were also inconsistencies in technical details related to ontology purposes, reasoners, connection mechanisms, or communications between CDSSs and EHR, EMR, or HIS systems. Inconsistent or missing information hampered reproducibility and further improvement of published work. We are obviously not the only group that has identified missing critical information as a problem in technical papers on similar topics [
Another missing piece is the evaluation and validation of ontologies or knowledge bases. Only 25% of publications mentioned that domain experts conducted evaluation or validation. A formal assessment or validation is critical to ensure the validity of the results from automated processes for some ontologies (or knowledge bases) derived from other automatic methods (eg, machine learning algorithms). Testing has not been conducted consistently across the publications. Some ontologies were authored by domain experts, which provides greater validity than those involving nondomain experts while constructing ontologies.
Thus, it is recommended that authors include essential technical details in publications. These technical details include CDSS application domains, intended CDSS users, CDSS notification types, CDSS evaluations (what, how, and by whom), CDSS rule sources, CDSS rule languages, CDSS rule engines, CDSS operation mechanisms, ontology use purposes, ontology sources (both content and code systems), ontology validation, reasoners, and connection or communication mechanisms between CDSSs and EMR, EHR, or HIS systems. Authors are highly encouraged to include such details to help readers reference, compare, and increase the reproducibility of the reported work.
Our review has limitations. Non-English publications or full-text unavailable publications were not included. Publications that focused only on CDSS rules [
When “CDSS” is not specified as a keyword, the search results may exclude publications. For example, our 2 previous papers [
The reuse, management, and maintenance of CDSS rules are critical yet challenging for their clinical application. Although ontologies have been used to contribute to the content of CDSS rules, they have not been used to facilitate CDSS rule reuse and sharing. Building a CDSS ontology, which could be the first tangible step, requires bridging high-level visions and operational efforts. Semantic interoperability remains a major challenge that must be overcome to achieve reuse of CDSS artifacts, including CDSS rules. The realization of semantic interoperability will not only allow for the reuse of CDSS artifacts, which are resource intensive to develop and maintain, but also provide practical insights to achieve interoperable patient records. This has been a long-lost aspect, and health care providers will be able to access patients’ complete records to provide safer and higher quality care every time to every patient. We believe that making CDSS rules interoperable can provide insightful guidance for interoperable patient records.
Incomplete technical details on CDSS rules and ontologies presented in publications should be addressed in future publications by including more detailed information about architectural diagrams; the mechanisms of connection among ontologies, CDSS rules, and EHR, EMR, or HIS systems; CDSS rule languages; reasoners; rule engines; the validation or authorization of ontologies and CDSS rules; the purposes of ontologies; ontology sources; and the management and maintenance of CDSS rules. Such information can help researchers to optimize design and development while also increasing reproducibility. Finally, the knowledge framework and the summarization of included publications are expected to guide future CDSS improvements and innovations, CDSS rules, and the integration and communication of CDSSs with EHR, EMR, or HIS systems.
Initial draft of codes and code groups used in reviewing/coding.
Refined version of codes and code groups used in reviewing/coding.
List of papers included in this systematic literature review (n=81).
PRISMA 2020 item checklist reported in our systematic literature review.
PRISMA 2020 for Abstracts checklist reported in our systematic literature review.
Word cloud generated from ATLAS.ti based on codes in the included publications .
Bar chart generated from ATLAS.ti showing the clinical domains of clinical decision support systems in included publications.
Basic clinical decision support system profiles in included publications (n=81).
Comparison of clinical decision support system rule characteristics in included publications (n=54).
Comparison of ontology roles in included publications (n=36).
Association for Computing Machinery
application programming interface
clinical decision support system
electronic health record
electronic medical record
hospital information system
Health Level 7
Nursing & Allied Health Database
Web Ontology Language
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
resource description framework
Systematized Nomenclature of Medicine-Clinical Terms
Semantic Query-Enhanced Web Rule Language
Semantic Web Rule Language
The authors thank professors James J Cimino, Dean F Sittig, and Adam Wright for their valuable feedback and suggestions on the manuscript. This work was supported by the National Institute of General Medical Sciences of the National Institutes of Health (under award number R01GM138589 and partially under award number P20GM121342). The content is solely the authors’ responsibility and does not necessarily represent the official views of the National Institutes of Health.
A request for additional data in addition to the results and appendices of this manuscript can be made to the corresponding author, and the final decision on data release will be made on a case-by-case basis, as appropriate.
XJ, HM, and YG designed the study, conducted the review and coding, drafted the first version, analyzed and interpreted the results, and revised the manuscript significantly. PB, DR, TL, CN, AF, LR, NH, and RG participated in the design of the study, analyzed and interpreted the results, and revised the manuscript significantly.
None declared.