We sought articles describing clinical decision support (CDS) systems (CDSSs) or risk prediction systems using FHIR standards. FHIR standards define resource types (ie, patients, medications, and clinical observations), data elements (ie, medication name and dosage), data formats (ie, JSON and XML files), and the use of standard ontologies (ie, Systematized Nomenclature of Medicine-Clinical Terms [SNOMED CT] and Logical Observation Identifiers Names and Codes [LOINC]), among others. Our initial search was performed on April 23, 2020, and given the progress of the field, it was updated again on October 11, 2022. Inclusion criteria involved all full-text articles published in English. We excluded abstracts, poster presentations, and meeting summaries. Embase, PubMed, and Web of Science were searched for cohort studies, case-control studies, and reviews. Our search terms for each database are found in Multimedia Appendix 2. Despite their increasing use by commercial entities, we did not search for commercial applications of FHIR, as their lack of peer review and limited reportability prevented a formal evaluation of their methods. Following the removal of duplicates, 153 articles were identified. Titles and abstracts were reviewed by 2 authors independently, with disagreements resolved by a third. Full-text articles that did not adequately describe system functionality, data sources, formats, security, performance, resource requirements, scalability, strengths, and limitations were excluded. We also excluded articles that described a model architecture using FHIR but did not incorporate it into a CDSS. A total of 39 full-text articles were included for full analysis.

Strengths and limitations of the applications were evaluated in terms of functionality, data sources, formats, security, performance, resource requirements, and scalability. Functionality was defined as the intended purpose of the algorithm and its capabilities, ranging from the integration of genomic data into the EHR [13,14] to CDSSs [15-17] and predictive models [18,19]. Data sources included information within electronic health care records and external sources, such as wearable devices [20]. Formats were evaluated based on system architecture and the technologies underlying the algorithms (eg, use of Bayesian networks [16], transformers [21], or rule-based methods [22]). Security was evaluated based on how the application handled sensitive health information, including encryption [23], use-and-access control mechanisms [24], or authorization platforms [25,26]. Performance and resource requirements refer to the processing time, memory, and computing needs of the applications. Finally, scalability refers to the likelihood of adoption by other health care systems or platforms (eg, use of open-source components [27] or cloud-based repositories [28]). Knowledge from the included articles was used to propose avenues of future development for optimizing machine learning–enabled systems.

CDSSs are algorithms that use health information to provide assistance for clinical decision-making tasks. Table 1 shows articles that focused on these support systems. Although many CDSSs lacked interoperability and external validity, several characteristics of CDSSs harbored potential for improving both efficacy and efficiency.

CDSS ontologies are a central tenant of CDSS interoperability. Generally, ontologies are a hierarchy of concepts that are defined by both a set of attributes and their relationships to other concepts, and they must meet several internal consistency and version control objectives [10]. Common ontologies include the SNOMED CT, LOINC, and National Cancer Institute Thesaurus (NCIT). Separate ontologies may conflict, such as in cases where models use different organizing principles, have varying degrees of granularity, or even exhibit contextual differences between clinical applications and biomedical research. Séroussi et al [34] faced this problem when creating a guideline for the optimal management of breast cancer by integrating a collection of pre-existing ontologies (NCIT and LOINC). They were able to resolve this conflict by using data visualization techniques and rules-based inference engines, though often their methods required the manual resolution of conflicts. Common ontologies can also omit essential elements. Dolin et al [13] were able to transform a library of drug-gene interactions into an FHIR standard to alert physicians when prescriptions are likely to cause adverse drug reactions. Specific disease classes may lack an interoperable ontology. For cancer, there are active efforts in the CodeX HL7 FHIR Accelerator community to capture oncologic data from the EHR by using the mCODE (minimal Common Oncology Data Elements) ontology [39,40].

Advanced CDSSs have been integrated with machine learning algorithms to process data, especially unstructured data, such as clinician notes. Gaebel et al [16] created a physician-facing CDSS that used Bayesian networks and medical language modules to identify the optimal management strategy for laryngeal cancer. Bayesian networks and other modeling approaches can estimate and infer unobserved but relevant variables, which is advantageous in representing complex diseases. Natural language processing is becoming an increasingly common tool. Hong et al [27], Semenov et al [20,33], and Whitaker et al [38] used semantic tags, rules-based extraction, and the scispaCy-based natural language processing pipeline to extract their concepts, though these methods require arduous labeling—the process of manually highlighting terms and classifying them—and lack validation on external data sets. Vocabulary and expressions often differ outside of the training context, requiring developers to further refine their language models after release by using test data and real-life examples.

Cloud-based solutions have made it possible to process large-scale and heterogeneous data and push the boundaries of CDSSs to encompass broader scenarios. El-Sappagh et al [28] developed a mobile app that integrates data from wearable monitors (eg, vital signs, physical activity, and blood glucose levels) with the EHR to provide recommendations for managing type 1 diabetes mellitus. The system delivers spoken education and lifestyle recommendations to patients’ mobile devices, using an ontology generated from clinical practice guidelines, expert opinions, and other published sources. Meanwhile, in countries with nationally integrated health systems, citizens may be able to assemble their data across different institutions by using a secure server, such as Azure [31]. Henry et al [18] created a real-time prediction system for critically ill patients that alerts staff to elevated sepsis risk and tracks trends in vitals by using cloud-based technology. In the outpatient setting, Kawamoto et al [30] incorporated data from several EHRs into an existing risk prediction model.

A total of 3 studies described visualization tools. Gordon et al [29] generated visual aids to show patients’ thrombocytopenia trends, along with computer-generated calculated panel reactive antibody levels, to facilitate the judicious use of platelet transfusions by physicians and blood banks, and Thayer et al [36] used translated FHIR concepts to graphically display a patient’s asthma history within a chart. Xiao et al [41] were able to use knowledge graph ontologies to map FHIR and Observational Medical Outcomes Partnership (OMOP) data standards.

Despite the considerable benefits of cloud-based systems, they can present additional security challenges. These range from traditional cybersecurity problems (including problems related to data security, access control, and the transmission of data over a network) to more CDSS-specific concerns (such as privacy leakage, whereby models can be queried by outside parties). HL7 FHIR has put forward specific security protocols in response to safety concerns, including the use of secure http communication channels, open authorization, and provenance (documentation of the origin, possession, and history of a piece of data) techniques, among others [42].

The rise in computing power and distributed system technologies facilitates general-purpose platforms that provide data standardization, data analysis, and model integration. Of the 39 included articles, 10 described FHIR-compliant data management and analytic platforms, as listed in Table 2. Although CDSSs require interoperability and multicenter clinical implementation, many clinical platforms did not support the real-time data integration that is necessary for clinical adoption.

Several papers addressed the challenge of integrating data from heterogeneous sources. Haarbrandt et al [24] proposed a platform that addresses this problem by developing techniques for converting disparate sources to FHIR standards prior to integration. The system is protected via fine-grained use-and-access control mechanisms that ensure secure data transmission among participating data sources. Metke-Jimenez et al [46] proposed an alternative approach to integrating several ontologies into a single web ontology language, allowing for updates to the ontology without changing the underlying data. For example, one could update the definition of sepsis and readily find all patients meeting the new definition. Distributed processing systems can be further enhanced via compartmentalization. Kopanitsa [44] and Semenov et al [47] developed a microservice platform that connects multiple systems via FHIR APIs. This platform was used to successfully deploy 400 CDSS models and 128 Bayesian diagnostic models in real time. Important to precision medicine, genomics data can now be linked to FHIR clinical data; 2 groups have created interoperability between the Variant Call Format for next-generation sequencing and FHIR [13,14].

Clinical information systems can aid in medical research, if properly designed. Although a prototype system proposed by Khalilia et al [25] ran 9 different machine learning models to generate data-driven, patient-level predictions, it lacked a researcher interface for the development and training of new models. In contrast, the KETOS platform proposed by Gruendner et al [14] allows researchers to request data sets, define cohorts, develop models, and deploy them as a web service. Both systems use Extract, Transform, Load pipelines to convert EHR data from their native format to the OMOP common data model format before storage. The KETOS platform’s comprehensive approach to data management and model deployment can aid researchers with limited backgrounds in data science.

Artificial intelligence clinical information systems depend on robust and secure APIs to interact with the clinical environment. APIs define quality and security standards for each type of interaction with external systems (eg, EHR systems, web browsers, and medical devices). Article summaries are shown in Table 3.

Of the included articles, 5 described auxiliary modules and APIs. Mandel et al [26] applied FHIR standards to the SMART platform, improving its interoperability by providing standard authentication, authorization, and profiling. The prototype genomics standard developed by Alterovitz et al [48], meanwhile, is currently in trial use to facilitate the consistent integration of clinical and genomic information through SMART-on-FHIR application. The application developers found the FHIR v4.0.1 specification easy to leverage, even without prior experience with FHIR.

Although FHIR has predefined resources and mechanisms for transmitting orders and values, methods for creating and validating orders are not predefined. To address this issue, Kopanitsa and Ivanov [51] proposed an FHIR-based mechanism for integrating laboratory and hospital information systems. The system generated laboratory orders, using the available tests in the laboratory information system, and prompted the user for relevant information (such as how many laboratory samples should be collected and when they should be collected). It is challenging to make clinical information systems both highly interoperable and secure without compromising data workflows. SecFHIR is an XML-based security approach to FHIR resources. Using schema permissions built into XML documents, Altamimi [23] generated robust security profiles that were context-aware (eg, privacy constraints can be overridden in emergency care situations).

Timely data availability is another barrier to implementing CDSSs in high-acuity environments. Kasparick et al [50] proposed a reference model to address the timeliness challenge by connecting medical devices to FHIR servers. This approach allows the APIs to function as data sources for predictive analytic and decision support systems. By using these methods, clinical information systems can maintain high interoperability and security without compromising data workflow. This has allowed for the development of disease-specific data hubs, which facilitate research on rare conditions or for reporting the results of COVID-19 polymerase chain reaction tests from disparate testing sites to a central authority [53,55]. CDS hooks are another technology that permit the integration of EHR data into external health care applications [57]. Used in collaboration with SMART on FHIR, CDS hooks are triggered by a specific action within the EHR (ie, ordering a medication). The CDS hooks then link the corresponding EHR data to an environment of decision support applications [58]. These CDS applications can then push recommendations in the form of “CDS cards” to the clinician. These technologies are currently being tested in real-word settings [59,60].

Although significant progress has been made in the field of FHIR data standards, this scoping review demonstrates that most CDSSs lack interoperability and actionable content. Several modules and APIs demonstrate the potential to enhance these systems, but they were not comprehensively integrated into the existing clinical workflows or were not validated on external patient populations. These limitations collectively reveal several opportunities to improve on existing methods to produce ideal clinical information systems, as illustrated in Figure 1.

Ideally, clinical information systems would function as innovation hubs for patient care and health care research. Due to the proprietary nature of hardware and software systems in institutions, infrastructure components (eg, data transformation, model development, authentication, and monitoring) are often painstakingly created de novo. Platforms, such as KETOS, however, can enable the sharing of core infrastructure, greatly accelerating the development and deployment of applications that are tailored to the needs of individual researchers, groups, and projects [14]. This “health care application development hub” would be shared for different applications to reuse models and for data processing and analyzing services.

Interoperability represents the goal of successful, cross-institutional sharing of data without additional, special effort. This remains in contrast to the current environment of fragmented data systems. Several elements of the current system impede progress toward integration and should be addressed. Sources of patient health information are numerous. At the point of data collection, clinicians may opt to store information in separate departments, erroneously duplicate patient descriptors, preferentially format or describe data, or use older data standards (HL7 v2 and v3). Further complexity may arise from the use of siloed systems, as exemplified by legacy systems built with local, stand-alone data conventions and incompatible ontologies [61]. The road to full interoperability is therefore paved with standards built to define, represent, transfer, and protect data as they travel between actors. Common communications standards, such as FHIR, have provided a useful framework for standardizing data transmission while maintaining semantic integrity at the patient level [15,56]. Importantly, these standards are built to mobilize data from legacy systems, making closely held data more publicly available [17]. By using OMOP-on-FHIR algorithms, pediatric data from Shriners Hospitals for Children can now be shared more widely by researchers [45]. More recently, 2 studies have examined the use of deep learning and transformer techniques to convert data elements in the EHR to interoperable FHIR standards, with subsequent application in prediction models [21,62]. Automation in data capture has the potential to reduce the costs and time associated with manual extraction.

Despite the benefits of an interoperable health data ecosystem, stakeholders are rarely incentivized to implement data standards. Organizational resistance to interoperability may stem from cultural differences, unfamiliarity with new technologies, or the fear that a newly adopted information-sharing standard may quickly become obsolete [63,64]. Among organizations, concerns regarding the loss of autonomy, a lack of trust, and the failure to realize financial gains impede interoperability and lead to so-called “information blocking.” The policies contained within the 21st Century Cures Act aim to improve information flow among actors in the system [65,66]. Apple, Google, and Samsung now have patient-facing health records that were developed along with FHIR standards to comply with these policies. In addition, while implementation models exist to help streamline the adoption of CDSSs, they contain important methodical flaws [67].

Unfamiliarity with interoperability standards may represent a substantial hurdle to adoption and subsequent interoperability. This challenge creates demand for subject matter experts who are familiar with the architecture, function, and implementation of data standards. Such experts must be able to anticipate the specific challenges of adapting their particular legacy systems to the interoperable standard but also recognize the benefit of successful adoption to guide organizational buy-in [68].

The need for timeliness in data sharing is driven both by data availability and by opportunities for real-time treatment support. An obvious example of this can be seen with continuous glucose monitoring units for patients with diabetes, which provide a regular source of data that can be implemented immediately to adjust insulin therapy [69,70].

System scalability is also essential to this task. Many of the systems evaluated in this review cannot scale in real time, as data volume or velocity increases dynamically (eg, processing 1000 patients in real time vs processing 100 patients in a static, retrospective training cohort). When scalability is impaired, predictions may not be delivered in time to augment clinical decision-making. Health Insurance Portability and Accountability Act (HIPAA)–compliant cloud platforms can scale allocated resources on demand. Therefore, optimal clinical information systems must offer scalability that is commensurate with the expected volume and velocity of data.

Each institution has policies that comply with municipal and federal security and privacy laws, making it challenging to share and aggregate data across multiple institutions. These challenges have been met with creative methods for aggregating multicenter data while maintaining patient privacy. One such method is to request only the minimum necessary information. This approach is emphasized heavily in the HIPAA and exemplified by El-Sappagh et al [28], who described a system that requests only the required EHR data elements for a specific patient. Other such mechanisms include authorization programs (enables specialized control over access to patient data), https, and WebSockets (Internet Engineering Task Force; provides secure communication over networks).

Alternatively, models can benefit from the knowledge derived from other data sets—usually in the form of model gradients or coefficients—without sharing the underlying data. This is known as federated learning—a system that trains on many local models with the same architecture and then aggregates the knowledge derived from each center into a global model (Figure 1). Although such an approach greatly reduces security and privacy risks by keeping the source records under the control of each local institution, even the gradients themselves pose a minor risk due to privacy leakage [28,71-75]. This risk, however, can be further reduced via the automated obfuscation of high-risk records or by adding noise to the gradients and coefficients before transmitting them to the central model. Given these advantages, federated learning is poised to supplant other methods for ensuring the data security and privacy of clinical information systems.

Finally, the recent explosion of large language models has raised further concerns regarding data privacy, as they are trained on clinical notes. This is an active field of study with multiple avenues for further research [76,77].

Machine learning–enabled clinical analytic and decision support systems have the potential to improve health care by automating standardized workflows and augmenting clinical decision-making. Nevertheless, most CDSSs lack interoperability and evidence of clinical utility. Common data models and interoperable data management platforms can address these limitations, but most intelligent clinical platforms are also compromised by the inadequate scalability for supporting real-time data processing. Existing clinical information systems could be improved by using foundational code infrastructures, common data models, and secure data processing and analytics on real-time platforms. Further progress in implementing these elements can generate information systems that improve care by helping patients, caregivers, and clinicians make effective, well-informed clinical decisions.