This paper is in the following e-collection/theme issue:

Reviews in Medical Informatics (82) Standards and Interoperability (60) Machine Learning (1370) Ontologies, Classifications, and Coding (223) Standards and Interoperability (60) Clinical Informatics (904) Clinical Information and Decision Making (1314) Machine Learning (1370) Ontologies, Classifications, and Coding (223) Standards and Interoperability (60) Clinical Informatics (904) Clinical Information and Decision Making (1314) Machine Learning (1370) Clinical Informatics (904) Clinical Information and Decision Making (1314) Ontologies, Classifications, and Coding (223)

Published on in Vol 11 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/48297, first published .
Machine Learning–Enabled Clinical Information Systems Using Fast Healthcare Interoperability Resources Data Standards: Scoping Review

Machine Learning–Enabled Clinical Information Systems Using Fast Healthcare Interoperability Resources Data Standards: Scoping Review

Machine Learning–Enabled Clinical Information Systems Using Fast Healthcare Interoperability Resources Data Standards: Scoping Review

Authors of this article:

Jeremy A Balch1, 2 Author Orcid Image ;   Matthew M Ruppert2, 3 Author Orcid Image ;   Tyler J Loftus1, 2 Author Orcid Image ;   Ziyuan Guan2, 3 Author Orcid Image ;   Yuanfang Ren2, 3 Author Orcid Image ;   Gilbert R Upchurch1 Author Orcid Image ;   Tezcan Ozrazgat-Baslanti2, 3 Author Orcid Image ;   Parisa Rashidi2, 4 Author Orcid Image ;   Azra Bihorac2, 3 Author Orcid Image
Article Authors Cited by (3) Tweetations (2) Metrics

    1Department of Surgery, University of Florida Health, , Gainesville, FL, , United States

    2Intelligent Critical Care Center, University of Florida, , Gainesville, FL, , United States

    3Department of Medicine, University of Florida, , Gainesville, FL, , United States

    4Department of Biomedical Engineering, University of Florida, , Gainesville, FL, , United States

    *these authors contributed equally

    Corresponding Author:

    Azra Bihorac, MSc, MD


    Background: Machine learning–enabled clinical information systems (ML-CISs) have the potential to drive health care delivery and research. The Fast Healthcare Interoperability Resources (FHIR) data standard has been increasingly applied in developing these systems. However, methods for applying FHIR to ML-CISs are variable.

    Objective: This study evaluates and compares the functionalities, strengths, and weaknesses of existing systems and proposes guidelines for optimizing future work with ML-CISs.

    Methods: Embase, PubMed, and Web of Science were searched for articles describing machine learning systems that were used for clinical data analytics or decision support in compliance with FHIR standards. Information regarding each system’s functionality, data sources, formats, security, performance, resource requirements, scalability, strengths, and limitations was compared across systems.

    Results: A total of 39 articles describing FHIR-based ML-CISs were divided into the following three categories according to their primary focus: clinical decision support systems (n=18), data management and analytic platforms (n=10), or auxiliary modules and application programming interfaces (n=11). Model strengths included novel use of cloud systems, Bayesian networks, visualization strategies, and techniques for translating unstructured or free-text data to FHIR frameworks. Many intelligent systems lacked electronic health record interoperability and externally validated evidence of clinical efficacy.

    Conclusions: Shortcomings in current ML-CISs can be addressed by incorporating modular and interoperable data management, analytic platforms, secure interinstitutional data exchange, and application programming interfaces with adequate scalability to support both real-time and prospective clinical applications that use electronic health record platforms with diverse implementations.

    JMIR Med Inform 2023;11:e48297

    doi:10.2196/48297

    Keywords

    ontologies; clinical decision support system; Fast Healthcare Interoperability Resources; FHIR; machine learning; ontology; interoperability; interoperable; decision support; information systems; review methodology; review methods; scoping review; clinical informatics 


    Data analytic tools provide essential contributions to scientific investigation and clinical decision-making [1]. These tools are in turn fueled by the volumes of data that have been generated since the passage of the Health Information Technology for Economic and Clinical Health Act in 2009, which incentivized the adoption of electronic health record (EHR) systems [2-4].

    EHR data, however, remain nonstandardized across institutions and, within an institution, may not be readily available for real-time analysis, thus impairing multi-institutional research efforts and care for individual patients across institutions [5-8]. The standards herein refer to the structure, organization, representation, and transmission of data. Health information exchange systems can mitigate these issues by using the Fast Healthcare Interoperability Resources (FHIR; pronounced “fire”) data standard [9]. The Health Level 7 (HL7) International standard developing organization sought to reduce the complexity of the HL7 version 3 Reference Information Model while maintaining semantic interoperability and thus adopted the FHIR standard in 2011 [10]. It supports multiple development platforms and has been embraced by major industry and government organizations. Since 2016, developers have engaged with Substitutable Medical Applications and Reusable Technologies (SMART) on FHIR to build EHR and commercial applications [11,12]. Despite the growth of technologies using FHIR standards, there is limited literature summarizing differences among machine learning–enabled clinical information systems (ML-CISs), and the best methods for applying FHIR remain unclear.

    This review describes the functionalities, strengths, and weaknesses of clinical applications that use the FHIR standard and have been described in the medical literature, and we propose guidelines for improved multi-institutional research initiatives and clinical applicability.


    Given the rapidly evolving nature of this field, we performed a scoping review to provide a critical appraisal of the current literature, with the goal of informing future studies. We followed the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines; the PRISMA-ScR checklist is available in Multimedia Appendix 1.

    Research Protocol

    We sought articles describing clinical decision support (CDS) systems (CDSSs) or risk prediction systems using FHIR standards. FHIR standards define resource types (ie, patients, medications, and clinical observations), data elements (ie, medication name and dosage), data formats (ie, JSON and XML files), and the use of standard ontologies (ie, Systematized Nomenclature of Medicine-Clinical Terms [SNOMED CT] and Logical Observation Identifiers Names and Codes [LOINC]), among others. Our initial search was performed on April 23, 2020, and given the progress of the field, it was updated again on October 11, 2022. Inclusion criteria involved all full-text articles published in English. We excluded abstracts, poster presentations, and meeting summaries. Embase, PubMed, and Web of Science were searched for cohort studies, case-control studies, and reviews. Our search terms for each database are found in Multimedia Appendix 2. Despite their increasing use by commercial entities, we did not search for commercial applications of FHIR, as their lack of peer review and limited reportability prevented a formal evaluation of their methods. Following the removal of duplicates, 153 articles were identified. Titles and abstracts were reviewed by 2 authors independently, with disagreements resolved by a third. Full-text articles that did not adequately describe system functionality, data sources, formats, security, performance, resource requirements, scalability, strengths, and limitations were excluded. We also excluded articles that described a model architecture using FHIR but did not incorporate it into a CDSS. A total of 39 full-text articles were included for full analysis.

    Article Evaluation

    Strengths and limitations of the applications were evaluated in terms of functionality, data sources, formats, security, performance, resource requirements, and scalability. Functionality was defined as the intended purpose of the algorithm and its capabilities, ranging from the integration of genomic data into the EHR [13,14] to CDSSs [15-17] and predictive models [18,19]. Data sources included information within electronic health care records and external sources, such as wearable devices [20]. Formats were evaluated based on system architecture and the technologies underlying the algorithms (eg, use of Bayesian networks [16], transformers [21], or rule-based methods [22]). Security was evaluated based on how the application handled sensitive health information, including encryption [23], use-and-access control mechanisms [24], or authorization platforms [25,26]. Performance and resource requirements refer to the processing time, memory, and computing needs of the applications. Finally, scalability refers to the likelihood of adoption by other health care systems or platforms (eg, use of open-source components [27] or cloud-based repositories [28]). Knowledge from the included articles was used to propose avenues of future development for optimizing machine learning–enabled systems.


    A total of 39 clinical tools that used FHIR standards were divided into the following three categories according to their primary focus: CDSSs (n=18), interoperable data management and analytic platforms (n=10), or auxiliary modules and application programming interfaces (APIs; n=11) that enhance ML-CISs.

    The CDSSs

    CDSSs are algorithms that use health information to provide assistance for clinical decision-making tasks. Table 1 shows articles that focused on these support systems. Although many CDSSs lacked interoperability and external validity, several characteristics of CDSSs harbored potential for improving both efficacy and efficiency.

    Table 1. Summary of intelligent clinical decision support systems.
    Source, yearFunctionalitiesStrengthsLimitations
    Curran et al [15], 2020Summarizes chronic obstructive pulmonary disease information, provides decision support, and suggests ordersDynamic embedding within EHRa and compatible with SMARTb-on-FHIRc submodulesLimited generalizability due to single center and single disease
    Dolin et al [13], 2018Uses drug-gene interaction data for clinical decision support triggered by EHR medication ordersAccesses a rules engine containing level A recommendations from a pharmacogenetics consortiumDifficult to query the rules engine for level A recommendations when triggered by EHR medication orders
    El-Sappagh et al [28], 2019Uses mobile health technologies to monitor and manage type 1 diabetesMost system processes are executed in the cloud; once configured, it runs on any EHR systemThe diabetes treatment ontology did not address emergency conditions and was not embedded within an EHR system
    Gaebel et al [16], 2016Generates digital patient models for clinical decision support for laryngeal cancerBayesian networks are well-suited for representing complex diseasesSystem architecture was described, but the system was not implemented clinically
    Gruender et al [14], 2019Combines next-generation sequencing genomics data with FHIR clinical dataOpen-source system that combines data formats and is portableManual data extraction and web-based filtering tool
    Gordon et al [29], 2017Displays patients’ thrombocytopenia trends along with computer-generated calculated panel reactive antibody levelsProvides real-time services and effective visual cuesData sources are limited
    Henry et al [18], 2018Predicts sepsis among intensive care unit patients in real timeCloud-based system that provides alerts to cliniciansPublic cloud-based solutions present safety issues
    Hong et al [27], 2019Phenotypes diabetes based on free-text notes and other structured dataConverts unstructured, semistructured, and structured data to appropriate FHIR componentsPerformance is not stable across different data sets
    Kawamoto et al [30], 2021Takes data from multiple EHRs and incorporates them into existing risk calculatorsPerformance measured with end user satisfaction studies and used existing application programming interfaceTested at a single institution and had data security concerns
    Park et al [31], 2022Personal health record application for employees, with links to health care resourcesFHIR-based cloud application that is applicable to multiple EHRs and provides secure access through AzureLimited integration of hospital data
    Schleyer et al [32], 2021Integrates selected data from statewide data systems into local EHRTranslates data from diverse sources into a common databaseExperience limited to a single EHR
    Semenov and Kopanitsa [20], 2018Recommends clinical decisions and actions based on EHR dataFree-text output for both physicians and patientsNo standard performance evaluation
    Semenov et al [33], 2018Recommends clinical decisions and actions based on EHR dataFree-text output for both physicians and patients and improved analytic workflow relative to prior versionsNo standard performance evaluation
    Séroussi et al [34], 2018Produces clinical practice guideline services for patients with breast cancerUses both data models and knowledge models and provides effective data analytic visualizationsImplemented on a small scale, proposed guidelines were not validated, and interguideline conflicts need to be resolved manually
    Tarumi et al [35], 2021Modeling of treatment outcomes for type 2 diabetesEffective use of SMART on FHIR for integration in local EHR, and design incorporated clinician feedbackNo external validation, limited access to cost data, and not yet compatible with all EHRs
    Thayer et al [36], 2021Automated graphical display of asthma historySmoothly integrated into EHRNot based on SMART, limiting interoperability
    Wang et al [37], 2019Comparison of machine learning algorithms for prediction of end-stage renal disease in type 2 diabetesExtraction of EHR data using FHIRSingle institution, no imputation of missing data, and no external validation
    Whitaker et al [38], 2022Machine learning algorithm to identify blood transfusion adverse eventsSynthesized structured and unstructured data from EHR to achieve reasonable accuracy compared to cliniciansRetrospective study more aligned toward research than clinical care

    aEHR: electronic health record.

    bSMART: Substitutable Medical Applications and Reusable Technologies.

    cFHIR: Fast Healthcare Interoperability Resources.

    CDSS ontologies are a central tenant of CDSS interoperability. Generally, ontologies are a hierarchy of concepts that are defined by both a set of attributes and their relationships to other concepts, and they must meet several internal consistency and version control objectives [10]. Common ontologies include the SNOMED CT, LOINC, and National Cancer Institute Thesaurus (NCIT). Separate ontologies may conflict, such as in cases where models use different organizing principles, have varying degrees of granularity, or even exhibit contextual differences between clinical applications and biomedical research. Séroussi et al [34] faced this problem when creating a guideline for the optimal management of breast cancer by integrating a collection of pre-existing ontologies (NCIT and LOINC). They were able to resolve this conflict by using data visualization techniques and rules-based inference engines, though often their methods required the manual resolution of conflicts. Common ontologies can also omit essential elements. Dolin et al [13] were able to transform a library of drug-gene interactions into an FHIR standard to alert physicians when prescriptions are likely to cause adverse drug reactions. Specific disease classes may lack an interoperable ontology. For cancer, there are active efforts in the CodeX HL7 FHIR Accelerator community to capture oncologic data from the EHR by using the mCODE (minimal Common Oncology Data Elements) ontology [39,40].

    Advanced CDSSs have been integrated with machine learning algorithms to process data, especially unstructured data, such as clinician notes. Gaebel et al [16] created a physician-facing CDSS that used Bayesian networks and medical language modules to identify the optimal management strategy for laryngeal cancer. Bayesian networks and other modeling approaches can estimate and infer unobserved but relevant variables, which is advantageous in representing complex diseases. Natural language processing is becoming an increasingly common tool. Hong et al [27], Semenov et al [20,33], and Whitaker et al [38] used semantic tags, rules-based extraction, and the scispaCy-based natural language processing pipeline to extract their concepts, though these methods require arduous labeling—the process of manually highlighting terms and classifying them—and lack validation on external data sets. Vocabulary and expressions often differ outside of the training context, requiring developers to further refine their language models after release by using test data and real-life examples.

    Cloud-based solutions have made it possible to process large-scale and heterogeneous data and push the boundaries of CDSSs to encompass broader scenarios. El-Sappagh et al [28] developed a mobile app that integrates data from wearable monitors (eg, vital signs, physical activity, and blood glucose levels) with the EHR to provide recommendations for managing type 1 diabetes mellitus. The system delivers spoken education and lifestyle recommendations to patients’ mobile devices, using an ontology generated from clinical practice guidelines, expert opinions, and other published sources. Meanwhile, in countries with nationally integrated health systems, citizens may be able to assemble their data across different institutions by using a secure server, such as Azure [31]. Henry et al [18] created a real-time prediction system for critically ill patients that alerts staff to elevated sepsis risk and tracks trends in vitals by using cloud-based technology. In the outpatient setting, Kawamoto et al [30] incorporated data from several EHRs into an existing risk prediction model.

    A total of 3 studies described visualization tools. Gordon et al [29] generated visual aids to show patients’ thrombocytopenia trends, along with computer-generated calculated panel reactive antibody levels, to facilitate the judicious use of platelet transfusions by physicians and blood banks, and Thayer et al [36] used translated FHIR concepts to graphically display a patient’s asthma history within a chart. Xiao et al [41] were able to use knowledge graph ontologies to map FHIR and Observational Medical Outcomes Partnership (OMOP) data standards.

    Despite the considerable benefits of cloud-based systems, they can present additional security challenges. These range from traditional cybersecurity problems (including problems related to data security, access control, and the transmission of data over a network) to more CDSS-specific concerns (such as privacy leakage, whereby models can be queried by outside parties). HL7 FHIR has put forward specific security protocols in response to safety concerns, including the use of secure http communication channels, open authorization, and provenance (documentation of the origin, possession, and history of a piece of data) techniques, among others [42].

    Data Management and Analytic Platforms

    The rise in computing power and distributed system technologies facilitates general-purpose platforms that provide data standardization, data analysis, and model integration. Of the 39 included articles, 10 described FHIR-compliant data management and analytic platforms, as listed in Table 2. Although CDSSs require interoperability and multicenter clinical implementation, many clinical platforms did not support the real-time data integration that is necessary for clinical adoption.

    Table 2. Summary of interoperable data management and analytic platforms.
    Source, yearFunctionalitiesStrengthsLimitations
    Gruendner et al [14], 2019Data analysis and model deployment in clinical environmentsApplied Docker virtualization that facilitates deployment across different environmentsPoor performance on Extract, Transform, Load processing; relatively inefficient (bottleneck) FHIRa transformation; and does not support real-time data processing
    Haarbrandt et al [24], 2018Integrating and transforming health data for oncology, cardiology, and infection controlOpen-source platform that allows for patient-level data sharingDoes not support real-time data processing
    Helm et al [43], 2022Builds interoperability between FHIR and BPMNbSupports BPMN clinical process models and improves explainabilityLacks some functionalities of the systems when used independently
    Khalilia et al [25], 2015Clinical predictive modeling using web services via HL7c FHIR standardsMaintains good performance across many different algorithmsDoes not support real-time data processing
    Kopanitsa [44], 2019Connects multiple health data systemsHas clear, effective workflowsDoes not support real-time data processing
    Marteau et al [45], 2022Increases availability of clinical pediatric data using OMOPd on FHIRImplementation across multiple local environmentsNot yet tested on real-word applications
    Metke-Jimenez et al [46], 2018Data searching, upgrading, and analyzing within multiple concept and category maps.Syndication models automatically update the dataDoes not support real-time data processing
    Semenov et al [47], 2019Clinical predictive analytics with text outputs to physicians and patientsProduces free-text outputs and graph visualizations pertaining to model recommendationsLimited support for real-time data processing.
    Thiess et al [17], 2022Application for support of shared decision-making in context of drug-drug interactionsEmbedded interoperability functions within modular CDSSe architecturesPerformance testing limited to electronic health record training module
    Xiao et al [41], 2022Enables FHIR and OMOP interoperability with generated clinical knowledge graphsSemantic foundation for development of explainable toolsFuture iterations will require expansion of mapping systems

    aFHIR: Fast Healthcare Interoperability Resources.

    bBPMN: Business Process Model and Notation 2.0.

    cHL7: Health Level 7.

    dOMOP: Observational Medical Outcomes Partnership.

    eCDSS: clinical decision support system.

    Several papers addressed the challenge of integrating data from heterogeneous sources. Haarbrandt et al [24] proposed a platform that addresses this problem by developing techniques for converting disparate sources to FHIR standards prior to integration. The system is protected via fine-grained use-and-access control mechanisms that ensure secure data transmission among participating data sources. Metke-Jimenez et al [46] proposed an alternative approach to integrating several ontologies into a single web ontology language, allowing for updates to the ontology without changing the underlying data. For example, one could update the definition of sepsis and readily find all patients meeting the new definition. Distributed processing systems can be further enhanced via compartmentalization. Kopanitsa [44] and Semenov et al [47] developed a microservice platform that connects multiple systems via FHIR APIs. This platform was used to successfully deploy 400 CDSS models and 128 Bayesian diagnostic models in real time. Important to precision medicine, genomics data can now be linked to FHIR clinical data; 2 groups have created interoperability between the Variant Call Format for next-generation sequencing and FHIR [13,14].

    Clinical information systems can aid in medical research, if properly designed. Although a prototype system proposed by Khalilia et al [25] ran 9 different machine learning models to generate data-driven, patient-level predictions, it lacked a researcher interface for the development and training of new models. In contrast, the KETOS platform proposed by Gruendner et al [14] allows researchers to request data sets, define cohorts, develop models, and deploy them as a web service. Both systems use Extract, Transform, Load pipelines to convert EHR data from their native format to the OMOP common data model format before storage. The KETOS platform’s comprehensive approach to data management and model deployment can aid researchers with limited backgrounds in data science.

    Auxiliary Modules and APIs

    Artificial intelligence clinical information systems depend on robust and secure APIs to interact with the clinical environment. APIs define quality and security standards for each type of interaction with external systems (eg, EHR systems, web browsers, and medical devices). Article summaries are shown in Table 3.

    Table 3. Summary of auxiliary modules and application programming interfaces (APIs).
    Source, yearFunctionalitiesStrengthsLimitations
    Altamimi [23], 2016Provide security for FHIRa functions to ensure patients’ privacyPolicies can be adjusted for circumstances (eg, emergency medical conditions can override privacy constructs)There is no description of a user-side module, which would be necessary for clinical application
    Alterovitz et al [48], 2015Link clinical and genomic data with an FHIR-compliant API for clinical decision supportEnsures consistent semantics in clinical data and handles multiple types of genomic dataEffects of clinical decision support apps on decision-making and outcomes were not reported.
    Dolin et al [49], 2021Variant Call Format–to–FHIR genomic standard converterReadily deployable to CDSSbLimited independent data analysis and does not support real-time data processing
    Kasparick et al [50], 2019Model an FHIR-compliant protocol for artificial intelligence–based systemsSupports multiple devices and multiple domains of dataNo clinical testing
    Kopanitsa and Ivanov [51], 2018FHIR-compliant APIs for data modelingHigh data exchanging efficiencyNo clinical testing
    Gabetta et al [52], 2021FHIR-on-OMOPc platform to support data storage and retrievalUse of standard OMOP vocabulariesNo clinical testing
    Guinez-Molinos et al [53], 2021Reports COVID-19 test results to central authorityInteroperable and portable; functionally verified with a pilot studyDeveloped using a predecessor system
    Mandel et al [26], 2016Updating an API platform with FHIR standardsImproves API interoperabilityEstablishes feasibility, but effects on clinical decision-making and outcomes are unknown.
    Rafee et al [54], 2022LOINCd-mapped core data set for eligibility screeningRapid EHRe screening for patient recruitmentRelied on expert labeling, which limits scalability
    Wood et al [55], 2021Allows sharing of patient data among care provision sites for hematologic disordersCompatible across EHRsFramework alone; awaiting evidence of implementation
    Yoo et al [56], 2022Method for integrating CDSS applications with EHRTransformation of EHR data into FHIR format for input into a reasoning engineNo validation of performance indices and usability of tested models

    aFHIR: Fast Healthcare Interoperability Resources.

    bCDSS: clinical decision support system.

    cOMOP: Observational Medical Outcomes Partnership.

    dLOINC: Logical Observation Identifiers Names and Codes.

    eEHR: electronic health record.

    Of the included articles, 5 described auxiliary modules and APIs. Mandel et al [26] applied FHIR standards to the SMART platform, improving its interoperability by providing standard authentication, authorization, and profiling. The prototype genomics standard developed by Alterovitz et al [48], meanwhile, is currently in trial use to facilitate the consistent integration of clinical and genomic information through SMART-on-FHIR application. The application developers found the FHIR v4.0.1 specification easy to leverage, even without prior experience with FHIR.

    Although FHIR has predefined resources and mechanisms for transmitting orders and values, methods for creating and validating orders are not predefined. To address this issue, Kopanitsa and Ivanov [51] proposed an FHIR-based mechanism for integrating laboratory and hospital information systems. The system generated laboratory orders, using the available tests in the laboratory information system, and prompted the user for relevant information (such as how many laboratory samples should be collected and when they should be collected). It is challenging to make clinical information systems both highly interoperable and secure without compromising data workflows. SecFHIR is an XML-based security approach to FHIR resources. Using schema permissions built into XML documents, Altamimi [23] generated robust security profiles that were context-aware (eg, privacy constraints can be overridden in emergency care situations).

    Timely data availability is another barrier to implementing CDSSs in high-acuity environments. Kasparick et al [50] proposed a reference model to address the timeliness challenge by connecting medical devices to FHIR servers. This approach allows the APIs to function as data sources for predictive analytic and decision support systems. By using these methods, clinical information systems can maintain high interoperability and security without compromising data workflow. This has allowed for the development of disease-specific data hubs, which facilitate research on rare conditions or for reporting the results of COVID-19 polymerase chain reaction tests from disparate testing sites to a central authority [53,55]. CDS hooks are another technology that permit the integration of EHR data into external health care applications [57]. Used in collaboration with SMART on FHIR, CDS hooks are triggered by a specific action within the EHR (ie, ordering a medication). The CDS hooks then link the corresponding EHR data to an environment of decision support applications [58]. These CDS applications can then push recommendations in the form of “CDS cards” to the clinician. These technologies are currently being tested in real-word settings [59,60].


    Key Findings

    Although significant progress has been made in the field of FHIR data standards, this scoping review demonstrates that most CDSSs lack interoperability and actionable content. Several modules and APIs demonstrate the potential to enhance these systems, but they were not comprehensively integrated into the existing clinical workflows or were not validated on external patient populations. These limitations collectively reveal several opportunities to improve on existing methods to produce ideal clinical information systems, as illustrated in Figure 1.

    ‎
    Figure 1. Sample model of a proposed machine learning–enabled clinical information system using FHIR data standards. AI: artificial intelligence; API: application programming interface; FHIR: Fast Healthcare Interoperability Resources; HL7: Health Level 7; IoT: Internet of Things; OMOP: Observational Medical Outcomes Partnership.

    Foundational Infrastructures Tailored to Individual Needs

    Ideally, clinical information systems would function as innovation hubs for patient care and health care research. Due to the proprietary nature of hardware and software systems in institutions, infrastructure components (eg, data transformation, model development, authentication, and monitoring) are often painstakingly created de novo. Platforms, such as KETOS, however, can enable the sharing of core infrastructure, greatly accelerating the development and deployment of applications that are tailored to the needs of individual researchers, groups, and projects [14]. This “health care application development hub” would be shared for different applications to reuse models and for data processing and analyzing services.

    Facilitating Interoperability Among Data Systems

    Interoperability represents the goal of successful, cross-institutional sharing of data without additional, special effort. This remains in contrast to the current environment of fragmented data systems. Several elements of the current system impede progress toward integration and should be addressed. Sources of patient health information are numerous. At the point of data collection, clinicians may opt to store information in separate departments, erroneously duplicate patient descriptors, preferentially format or describe data, or use older data standards (HL7 v2 and v3). Further complexity may arise from the use of siloed systems, as exemplified by legacy systems built with local, stand-alone data conventions and incompatible ontologies [61]. The road to full interoperability is therefore paved with standards built to define, represent, transfer, and protect data as they travel between actors. Common communications standards, such as FHIR, have provided a useful framework for standardizing data transmission while maintaining semantic integrity at the patient level [15,56]. Importantly, these standards are built to mobilize data from legacy systems, making closely held data more publicly available [17]. By using OMOP-on-FHIR algorithms, pediatric data from Shriners Hospitals for Children can now be shared more widely by researchers [45]. More recently, 2 studies have examined the use of deep learning and transformer techniques to convert data elements in the EHR to interoperable FHIR standards, with subsequent application in prediction models [21,62]. Automation in data capture has the potential to reduce the costs and time associated with manual extraction.

    Overcoming Organizational Resistance to Interoperability Standards

    Despite the benefits of an interoperable health data ecosystem, stakeholders are rarely incentivized to implement data standards. Organizational resistance to interoperability may stem from cultural differences, unfamiliarity with new technologies, or the fear that a newly adopted information-sharing standard may quickly become obsolete [63,64]. Among organizations, concerns regarding the loss of autonomy, a lack of trust, and the failure to realize financial gains impede interoperability and lead to so-called “information blocking.” The policies contained within the 21st Century Cures Act aim to improve information flow among actors in the system [65,66]. Apple, Google, and Samsung now have patient-facing health records that were developed along with FHIR standards to comply with these policies. In addition, while implementation models exist to help streamline the adoption of CDSSs, they contain important methodical flaws [67].

    Hiring Specialists to Manage Standards Adoption

    Unfamiliarity with interoperability standards may represent a substantial hurdle to adoption and subsequent interoperability. This challenge creates demand for subject matter experts who are familiar with the architecture, function, and implementation of data standards. Such experts must be able to anticipate the specific challenges of adapting their particular legacy systems to the interoperable standard but also recognize the benefit of successful adoption to guide organizational buy-in [68].

    Timely Data Acquisition

    The need for timeliness in data sharing is driven both by data availability and by opportunities for real-time treatment support. An obvious example of this can be seen with continuous glucose monitoring units for patients with diabetes, which provide a regular source of data that can be implemented immediately to adjust insulin therapy [69,70].

    System scalability is also essential to this task. Many of the systems evaluated in this review cannot scale in real time, as data volume or velocity increases dynamically (eg, processing 1000 patients in real time vs processing 100 patients in a static, retrospective training cohort). When scalability is impaired, predictions may not be delivered in time to augment clinical decision-making. Health Insurance Portability and Accountability Act (HIPAA)–compliant cloud platforms can scale allocated resources on demand. Therefore, optimal clinical information systems must offer scalability that is commensurate with the expected volume and velocity of data.

    Minimizing Discoverable Patient Data

    Each institution has policies that comply with municipal and federal security and privacy laws, making it challenging to share and aggregate data across multiple institutions. These challenges have been met with creative methods for aggregating multicenter data while maintaining patient privacy. One such method is to request only the minimum necessary information. This approach is emphasized heavily in the HIPAA and exemplified by El-Sappagh et al [28], who described a system that requests only the required EHR data elements for a specific patient. Other such mechanisms include authorization programs (enables specialized control over access to patient data), https, and WebSockets (Internet Engineering Task Force; provides secure communication over networks).

    Alternatively, models can benefit from the knowledge derived from other data sets—usually in the form of model gradients or coefficients—without sharing the underlying data. This is known as federated learning—a system that trains on many local models with the same architecture and then aggregates the knowledge derived from each center into a global model (Figure 1). Although such an approach greatly reduces security and privacy risks by keeping the source records under the control of each local institution, even the gradients themselves pose a minor risk due to privacy leakage [28,71-75]. This risk, however, can be further reduced via the automated obfuscation of high-risk records or by adding noise to the gradients and coefficients before transmitting them to the central model. Given these advantages, federated learning is poised to supplant other methods for ensuring the data security and privacy of clinical information systems.

    Finally, the recent explosion of large language models has raised further concerns regarding data privacy, as they are trained on clinical notes. This is an active field of study with multiple avenues for further research [76,77].

    Conclusions

    Machine learning–enabled clinical analytic and decision support systems have the potential to improve health care by automating standardized workflows and augmenting clinical decision-making. Nevertheless, most CDSSs lack interoperability and evidence of clinical utility. Common data models and interoperable data management platforms can address these limitations, but most intelligent clinical platforms are also compromised by the inadequate scalability for supporting real-time data processing. Existing clinical information systems could be improved by using foundational code infrastructures, common data models, and secure data processing and analytics on real-time platforms. Further progress in implementing these elements can generate information systems that improve care by helping patients, caregivers, and clinicians make effective, well-informed clinical decisions.

    Acknowledgments

    We would like to thank Zachary Hodges, Zhang Feng, and Shounak Datta for their initial contributions.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist for scoping reviews.

    PDF File, 599 KB
    Multimedia Appendix 2

    Search criteria.

    DOCX File, 12 KB
    1. Centers for Disease Control and Prevention, National Center for Health Statistics. Early release of selected estimates based on data from the 2018 National Health Interview Survey. URL: www.cdc.gov/nchs/nhis/releases/released201905.htm [accessed 2023-07-28]
    2. Birkhead GS, Klompas M, Shah NR. Uses of electronic health records for public health surveillance to advance public health. Annu Rev Public Health 2015 Mar 18;36:345-359 [CrossRef] [Medline]
    3. Adler-Milstein J, Holmgren AJ, Kralovec P, Worzala C, Searcy T, Patel V. Electronic health record adoption in US hospitals: the emergence of a digital “advanced use” divide. J Am Med Inform Assoc 2017 Nov 1;24(6):1142-1148 [CrossRef] [Medline]
    4. Stanford Medicine. Stanford Medicine 2017 health trends report: harnessing the power of data in health. 2017. URL: med.stanford.edu/content/dam/sm/sm-news/documents/StanfordMedicineHealthTrendsWhitePaper2017.pdf [accessed 2023-07-28]
    5. Unni N, Peddinghaus M, Tormey CA, Stack G. Record fragmentation due to transfusion at multiple health care facilities: a risk factor for delayed hemolytic transfusion reactions. Transfusion 2014 Jan;54(1):98-103 [CrossRef] [Medline]
    6. Hempstead K, Delia D, Cantor JC, Nguyen T, Brenner J. The fragmentation of hospital use among a cohort of high utilizers: implications for emerging care coordination strategies for patients with multiple chronic conditions. Med Care 2014 Mar;52(Suppl 3):S67-S74 [CrossRef] [Medline]
    7. Justiniano CF, Xu Z, Becerra AZ, Aquina CT, Boodry CI, Swanger AA, et al. Surgeon care fragmentation during readmission after colorectal surgery is associated with increased mortality: continuity of care counts. J Am Coll Surg 2017;225(4):S126-S127 [CrossRef]
    8. Tsai TC, Orav EJ, Jha AK. Care fragmentation in the postdischarge period: surgical readmissions, distance of travel, and postoperative mortality. JAMA Surg 2015 Jan;150(1):59-64 [CrossRef] [Medline]
    9. HL7 International. FHIR V5.0.0. URL: hl7.org/fhir [accessed 2023-07-28]
    10. Shortliffe EH, Cimino JJ. Biomedical Informatics: Computer Applications in Health Care and Biomedicine. Cham, Switzerland: Springer; 2021. [CrossRef]
    11. Payne TH, Corley S, Cullen TA, Gandhi TK, Harrington L, Kuperman GJ, et al. Report of the AMIA EHR-2020 Task Force on the status and future direction of EHRs. J Am Med Inform Assoc 2015 Sep;22(5):1102-1110 [CrossRef] [Medline]
    12. Griffin AC, He L, Sunjaya AP, King AJ, Khan Z, Nwadiugwu M, et al. Clinical, technical, and implementation characteristics of real-world health applications using FHIR. JAMIA Open 2022 Oct 12;5(4):ooac077 [CrossRef] [Medline]
    13. Dolin RH, Boxwala A, Shalaby J. A pharmacogenomics clinical decision support service based on FHIR and CDS hooks. Methods Inf Med 2018 Dec;57(S 02):e115-e123 [CrossRef] [Medline]
    14. Gruendner J, Schwachhofer T, Sippl P, Wolf N, Erpenbeck M, Gulden C, et al. KETOS: clinical decision support and machine learning as a service - A training and deployment platform based on Docker, OMOP-CDM, and FHIR web services. PLoS One 2019 Oct 3;14(10):e0223010 [CrossRef] [Medline]
    15. Curran RL, Kukhareva PV, Taft T, Weir CR, Reese TJ, Nanjo C, et al. Integrated displays to improve chronic disease management in ambulatory care: a SMART on FHIR application informed by mixed-methods user testing. J Am Med Inform Assoc 2020 Aug 1;27(8):1225-1234 [CrossRef] [Medline]
    16. Gaebel J, Cypko MA, Lemke HU. Accessing patient information for probabilistic patient models using existing standards. Stud Health Technol Inform 2016;223:107-112 [Medline]
    17. Thiess H, Del Fiol G, Malone DC, Cornia R, Sibilla M, Rhodes B, et al. Coordinated use of Health Level 7 standards to support clinical decision support: case study with shared decision making and drug-drug interactions. Int J Med Inform 2022 Mar 21;162:104749 [CrossRef] [Medline]
    18. Henry JR, Lynch D, Mals J, Shashikumar SP, Holder A, Sharma A, et al. A FHIR-enabled streaming sepsis prediction system for ICUs. Annu Int Conf IEEE Eng Med Biol Soc 2018 Jul;2018:4093-4096 [CrossRef] [Medline]
    19. Wang S, Han J, Jung SY, Oh TJ, Yao S, Lim S, et al. Development and implementation of patient-level prediction models of end-stage renal disease for type 2 diabetes patients using fast healthcare interoperability resources. Sci Rep 2022 Jul 4;12(1):11232 [CrossRef] [Medline]
    20. Semenov I, Kopanitsa G. Decision support system based on FHIR profiles. Stud Health Technol Inform 2018;249:117-121 [Medline]
    21. Sun H, Depraetere K, Meesseman L, De Roo J, Vanbiervliet M, De Baerdemaeker J, et al. A scalable approach for developing clinical risk prediction applications in different hospitals. J Biomed Inform 2021 Jun;118:103783 [CrossRef] [Medline]
    22. Iglesias N, Juarez JM, Campos M. Comprehensive analysis of rule formalisms to represent clinical guidelines: selection criteria and case study on antibiotic clinical guidelines. Artif Intell Med 2020 Mar;103:101741 [CrossRef] [Medline]
    23. Altamimi AM. SecFHIR: a security specification model for fast healthcare interoperability resources. Int J Adv Comput Sci Appl 2016 Jun;7(6):350-355 [CrossRef]
    24. Haarbrandt B, Schreiweis B, Rey S, Sax U, Scheithauer S, Rienhoff O, et al. HiGHmed - an open platform approach to enhance care and research across institutional boundaries. Methods Inf Med 2018 Jul;57(S 01):e66-e81 [CrossRef] [Medline]
    25. Khalilia M, Choi M, Henderson A, Iyengar S, Braunstein M, Sun J. Clinical predictive modeling development and deployment through FHIR web services. AMIA Annu Symp Proc 2015 Nov 5;2015:717-726 [Medline]
    26. Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc 2016 Sep;23(5):899-908 [CrossRef] [Medline]
    27. Hong N, Wen A, Shen F, Sohn S, Wang C, Liu H, et al. Developing a scalable FHIR-based clinical data normalization pipeline for standardizing and integrating unstructured and structured electronic health record data. JAMIA Open 2019 Oct 18;2(4):570-579 [CrossRef] [Medline]
    28. El-Sappagh S, Ali F, Hendawi A, Jang JH, Kwak KS. A mobile health monitoring-and-treatment system based on integration of the SSN sensor ontology and the HL7 FHIR standard. BMC Med Inform Decis Mak 2019 May 10;19(1):97 [CrossRef] [Medline]
    29. Gordon WJ, Baronas J, Lane WJ. A FHIR human leukocyte antigen (HLA) interface for platelet transfusion support. Appl Clin Inform 2017 Jun 7;8(2):603-611 [CrossRef] [Medline]
    30. Kawamoto K, Kukhareva PV, Weir C, Flynn MC, Nanjo CJ, Martin DK, et al. Establishing a multidisciplinary initiative for interoperable electronic health record innovations at an academic medical center. JAMIA Open 2021 Jul 31;4(3):ooab041 [CrossRef] [Medline]
    31. Park C, You SC, Jeon H, Jeong CW, Choi JW, Park RW. Development and validation of the Radiology Common Data Model (R-CDM) for the international standardization of medical imaging data. Yonsei Med J 2022 Jan;63(Suppl):S74-S83 [CrossRef] [Medline]
    32. Schleyer T, Williams L, Gottlieb J, Weaver C, Saysana M, Azar J, et al. The Indiana Learning Health System Initiative: early experience developing a collaborative, regional learning health system. Learn Health Syst 2021 Jun 23;5(3):e10281 [CrossRef] [Medline]
    33. Semenov I, Kopanitsa G, Denisov D, Alexandr Y, Osenev R, Andreychuk Y. Patients decision aid system based on FHIR profiles. J Med Syst 2018 Jul 31;42(9):166 [CrossRef] [Medline]
    34. Séroussi B, Guézennec G, Lamy JB, Muro N, Larburu N, Sekar BD, et al. Reconciliation of multiple guidelines for decision support: a case study on the multidisciplinary management of breast cancer within the DESIREE project. AMIA Annu Symp Proc 2018 Apr 16;2017:1527-1536 [Medline]
    35. Tarumi S, Takeuchi W, Chalkidis G, Rodriguez-Loya S, Kuwata J, Flynn M, et al. Leveraging artificial intelligence to improve chronic disease care: methods and application to pharmacotherapy decision support for type-2 diabetes mellitus. Methods Inf Med 2021 Jun;60(S 01):e32-e43 [CrossRef] [Medline]
    36. Thayer JG, Ferro DF, Miller JM, Karavite D, Grundmeier RW, Utidjian L, et al. Human-centered development of an electronic health record-embedded, interactive information visualization in the emergency department using fast healthcare interoperability resources. J Am Med Inform Assoc 2021 Jul 14;28(7):1401-1410 [CrossRef] [Medline]
    37. Wang Z, Song M, Zhang Z, Song Y, Wang Q, Qi H. Beyond Inferring class representatives: user-level privacy leakage from Federated learning. Presented at: IEEE INFOCOM 2019 - IEEE Conference on Computer Communications; Paris, France p. 2512-2520 [CrossRef]
    38. Whitaker B, Pizarro J, Deady M, Williams A, Ezzeldin H, Belov A, et al. Detection of allergic transfusion-related adverse events from electronic medical records. Transfusion 2022 Oct;62(10):2029-2038 [CrossRef] [Medline]
    39. HL7 International. Minimal Common Oncology Data Elements (mCODE) implementation guide. URL: hl7.org/fhir/us/mcode [accessed 2023-07-31]
    40. HL7 International. CodeX. URL: www.hl7.org/codex [accessed 2023-07-31]
    41. Xiao G, Pfaff E, Prud’hommeaux E, Booth D, Sharma DK, Huo N, et al. FHIR-Ontop-OMOP: building clinical knowledge graphs in FHIR RDF with the OMOP Common Data Model. J Biomed Inform 2022 Oct;134:104201 [CrossRef] [Medline]
    42. HL7 International. 6.1.0 FHIR security. URL: build.fhir.org/security.html [accessed 2023-07-31]
    43. Helm E, Pointner A, Krauss O, Schuler A, Traxler B, Arthofer K, et al. FHIR2BPMN: delivering actionable knowledge by transforming between clinical pathways and executable models. Stud Health Technol Inform 2022 May 16;292:9-14 [CrossRef] [Medline]
    44. Kopanitsa G. Microservice architecture to provide medical data management for decision support. Stud Health Technol Inform 2019;261:230-235 [Medline]
    45. Marteau BL, Zhu Y, Giuste F, Shi W, Carpenter A, Hilton C, et al. Accelerating multi-site health informatics with streamlined data infrastructure using OMOP-on-FHIR. July 11-15, 2022 Presented at: 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC); Glasgow, Scotland
    46. Metke-Jimenez A, Steel J, Hansen D, Lawley M. Ontoserver: a syndicated terminology server. J Biomed Semantics 2018 Sep 17;9(1):24 [CrossRef] [Medline]
    47. Semenov I, Osenev R, Gerasimov S, Kopanitsa G, Denisov D, Andreychuk Y. Experience in developing an FHIR medical data management platform to provide clinical decision support. Int J Environ Res Public Health 2019 Dec 20;17(1):73 [CrossRef] [Medline]
    48. Alterovitz G, Warner J, Zhang P, Chen Y, Ullman-Cullere M, Kreda D, et al. SMART on FHIR genomics: facilitating standardized clinico-genomic apps. J Am Med Inform Assoc 2015 Nov;22(6):1173-1178 [CrossRef] [Medline]
    49. Dolin RH, Gothi SR, Boxwala A, Heale BSE, Husami A, Jones J, et al. vcf2fhir: a utility to convert VCF files into Hl7 FHIR format for genomics-EHR integration. BMC Bioinformatics 2021 Mar 2;22(1):104 [CrossRef] [Medline]
    50. Kasparick M, Andersen B, Franke S, Rockstroh M, Golatowski F, Timmermann D, et al. Enabling artificial intelligence in high acuity medical environments. Minim Invasive Ther Allied Technol 2019 Apr;28(2):120-126 [CrossRef] [Medline]
    51. Kopanitsa G, Ivanov A. Implementation of Fast Healthcare Interoperability Resources for an integration of laboratory and hospital information systems. Stud Health Technol 2018;247:11-15 [Medline]
    52. Gabetta M, Alloni A, Polce F, Lanzola G, Parimbelli E, Barbarini N. Development of a FHIR layer on top of the OMOP Common Data Model for the CAPABLE project. Stud Health Technol Inform 2021 Nov 18;287:28-29 [CrossRef] [Medline]
    53. Guinez-Molinos S, Andrade JM, Negrete AM, Vidal SE, Rios E. Interoperable platform to report polymerase chain reaction SARS-CoV-2 tests from laboratories to the Chilean government: development and implementation study. JMIR Med Inform 2021 Jan 20;9(1):e25149 [CrossRef] [Medline]
    54. Rafee A, Riepenhausen S, Neuhaus P, Meidt A, Dugas M, Varghese J. ELaPro, a LOINC-mapped core dataset for top laboratory procedures of eligibility screening for clinical trials. BMC Med Res Methodol 2022 May 14;22(1):141 [CrossRef] [Medline]
    55. Wood WA, Marks P, Plovnick RM, Hewitt K, Neuberg DS, Walters S, et al. ASH Research Collaborative: a real-world data infrastructure to support real-world evidence development and learning healthcare systems in hematology. Blood Adv 2021 Dec 14;5(23):5429-5438 [CrossRef] [Medline]
    56. Yoo J, Lee J, Min JY, Choi SW, Kwon JM, Cho I, et al. Development of an interoperable and easily transferable clinical decision support system deployment platform: system design and development study. J Med Internet Res 2022 Jul 27;24(7):e37928 [CrossRef] [Medline]
    57. CDS hooks. URL: cds-hooks.org [accessed 2023-07-31]
    58. Strasberg HR, Rhodes B, Del Fiol G, Jenders RA, Haug PJ, Kawamoto K. Contemporary clinical decision support standards using Health Level Seven International Fast Healthcare Interoperability Resources. J Am Med Inform Assoc 2021 Jul 30;28(8):1796-1806 [CrossRef] [Medline]
    59. Jung S, Bae S, Seong D, Oh OH, Kim Y, Yi BK. Shared Interoperable clinical decision support service for drug-allergy interaction checks: implementation study. JMIR Med Inform 2022 Nov 10;10(11):e40338 [CrossRef] [Medline]
    60. Morgan KL, Kukhareva PV, Warner PB, Wilkof J, Snyder M, Horton D, et al. Using CDS hooks to increase SMART on FHIR app utilization: a cluster-randomized trial. J Am Med Inform Assoc 2022 Aug 16;29(9):1461-1470 [CrossRef] [Medline]
    61. Olaronke I, Soriyan A, Gambo I, Olaleke J. Interoperability in healthcare: benefits, challenges and resolutions. Int J Innov Appl Stud 2013 May;3(1):262-270
    62. Rajkomar A, Oren E, Chen K, Dai AM, Hajaj N, Hardt M, et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Med 2018 May 8;1:18 [CrossRef] [Medline]
    63. Han L, Liu J, Evans R, Song Y, Ma J. Factors influencing the adoption of health information standards in health care organizations: a systematic review based on best fit framework synthesis. JMIR Med Inform 2020 May 15;8(5):e17334 [CrossRef] [Medline]
    64. Henning F. Adoption of Interoperability standards in government information networks: an initial framework of influence factors. Presented at: ICEGOV ’13: 7th International Conference on Theory and Practice of Electronic Governance; Seoul Republic of Korea p. 264-267 [CrossRef]
    65. Everson J, Patel V, Adler-Milstein J. Information blocking remains prevalent at the start of 21st Century Cures Act: results from a survey of health information exchange organizations. J Am Med Inform Assoc 2021 Mar 18;28(4):727-732 [CrossRef] [Medline]
    66. Health and Human Services Department. 21st Century Cures Act: Interoperability, information blocking, and the ONC Health IT Certification Program. 2020. URL: www.federalregister.gov/​documents/​2020/​05/​01/​2020-07419/​21st-century-cures-act-interoperability-information-blocking-and-the-onc-health-it-certification [accessed 2023-07-31]
    67. Elwyn G, Scholl I, Tietbohl C, Mann M, Edwards AGK, Clay C, et al. “Many miles to go …”: a systematic review of the implementation of patient decision support interventions into routine clinical practice. BMC Med Inform Decis Mak 2013 Nov 29;13 Suppl 2(Suppl 2):S14 [CrossRef] [Medline]
    68. Lamprinakos GC, Mousas AS, Kapsalis AP, Kaklamani DI, Venieris IS, Boufis AD, et al. Using FHIR to develop a healthcare mobile application. November 3-5, 2014 Presented at: 2014 4th International Conference on Wireless Mobile Communication and Healthcare - Transforming Healthcare Through Innovations in Mobile and Wireless Technologies (MOBIHEALTH); Athens, Greece [CrossRef]
    69. Eberle C, Stichling S, Löhnert M. Diabetology 4.0: scoping review of novel insights and possibilities offered by digitalization. J Med Internet Res 2021 Mar 24;23(3):e23475 [CrossRef] [Medline]
    70. Hommel E, Olsen B, Battelino T, Conget I, Schütz-Fuhrmann I, Hoogma R, et al. Impact of continuous glucose monitoring on quality of life, treatment satisfaction, and use of medical care resources: analyses from the SWITCH study. Acta Diabetol 2014 Oct;51(5):845-851 [CrossRef] [Medline]
    71. Hitaj B, Ateniese G, Perez-Cruz F. Deep models under the GAN: information leakage from collaborative deep learning. Presented at: CCS ’17: 2017 ACM SIGSAC Conference on Computer and Communications Security; Dallas, TX p. 603-618 [CrossRef]
    72. Kairouz P, McMahan HB, Avent B, Bellet A, Bennis M, Bhagoji AN, et al. Advances and Open Problems in Federated Learning. Norwell, MA: Now Foundations and Trends; 2021. [CrossRef]
    73. Nasr M, Shokri R, Houmansadr A. Comprehensive privacy analysis of deep learning: passive and active white-box inference attacks against centralized and federated learning. May 19-23, 2019 Presented at: 2019 IEEE Symposium on Security and Privacy (SP); San Francisco, CA. [CrossRef]
    74. Phong LT, Aono Y, Hayashi T, Wang L, Moriai S. Privacy-preserving deep learning: revisited and enhanced. June 6-7, 2017 Presented at: 8th International Conference on Applications and Techniques in Information Security; Auckland, New Zealand [CrossRef]
    75. Wei W, Liu L, Loper M, Chow KH, Gursoy ME, Truex S, et al. A framework for evaluating gradient leakage attacks in federated learning. arXiv. Preprint posted online on April 23, 2020 [CrossRef]
    76. Sebastian G. Do ChatGPT and other AI chatbots pose a cybersecurity risk?: an exploratory study. International Journal of Security and Privacy in Pervasive Computing 2023 Jan;15(1):1-11 [CrossRef]
    77. Yoon HJ, Stanley C, Christian JB, Klasky HB, Blanchard AE, Durbin EB, et al. Optimal vocabulary selection approaches for privacy-preserving deep NLP model training for information extraction and cancer epidemiology. Cancer Biomark 2022;33(2):185-198 [CrossRef] [Medline]

    ‎
    API: application programming interface
    CDS: clinical decision support
    CDSS: clinical decision support system
    EHR: electronic health record
    FHIR: Fast Healthcare Interoperability Resources
    HIPAA: Health Insurance Portability and Accountability Act
    HL7: Health Level 7
    LOINC: Logical Observation Identifiers Names and Codes
    mCODE: minimal Common Oncology Data Elements
    ML-CIS: machine learning–enabled clinical information system
    NCIT: National Cancer Institute Thesaurus
    OMOP: Observational Medical Outcomes Partnership
    PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews
    SMART: Substitutable Medical Applications and Reusable Technologies
    SNOMED CT: Systematized Nomenclature of Medicine-Clinical Terms

    Edited by Christian Lovis; submitted 19.04.23; peer-reviewed by George Sakellaropoulos, Glorin Sebastian; final revised version received 15.06.23; accepted 17.06.23; published 24.08.23

    Copyright

    © Jeremy A Balch, Matthew M Ruppert, Tyler J Loftus, Ziyuan Guan, Yuanfang Ren, Gilbert R Upchurch, Tezcan Ozrazgat-Baslanti, Parisa Rashidi, Azra Bihorac. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 24.8.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on https://medinform.jmir.org/, as well as this copyright and license information must be included.