Electronic health records provide digital means to store and retrieve clinical data for millions of patients. Leveraging these real-world data provides an exciting opportunity for health care research [1]. However, the heterogeneity of electronic health record systems often results in interoperability challenges, hindering comprehensive analysis—especially across institutional boundaries [2,3]. This is where the Health Level Seven Fast Healthcare Interoperability Resources (FHIR) can help by providing a standardized framework for modeling and exchanging health care data [4]. The standard describes a data model consisting of resources that represent concepts in the health care domain. These resources are connected via references and form a graph. For example, a FHIR “Condition” resource representing a medical diagnosis may reference a “Patient” resource to which it applies and a “Practitioner” resource identifying the person recording the diagnosis. The FHIR standard also defines an HTTP representational state transfer (REST) interface and its semantics. This interface is implemented by FHIR servers to support the creation, reading, updating, and deletion of resources. Additionally, FHIR Search provides a comprehensive framework for filtering and retrieving relevant resources [5].

The increased adoption of FHIR is driving its use for analytic use cases, including decision support, feasibility research, and epidemiological research [6,7]. When conducting analytical research on FHIR data, familiarity with the graphlike structure, conventions, data types, and search semantics is required. From a purely syntactic perspective, data science and machine learning are usually conducted on tabular data, whereas FHIR resources are nested and interconnected—often encoded as JSON or XML file formats. Due to this gap, several projects aim to convert FHIR resources to a structure more suitable for analytical use cases. These approaches either interact with FHIR servers in a combination of FHIR Search and local postprocessing [8-11] or encode and store resources in a relational or columnar format first [12-15]. Nevertheless, the performance implications of these methods remain largely unexplored.

In this work, we compared analytical approaches that operate on Health Level Seven FHIR as the underlying data representation but differ in how the data are accessed and processed, ranging from REST-based querying of FHIR servers to SQL- and Spark-based querying of tabular-encoded FHIR datasets.

There are at least 5 programming libraries that aim to implement analytical features using the FHIR Search API to retrieve data [8-11]. This is indicative not only of the increased use of FHIR as a format for conducting clinical research but also of an attempt at leveraging well-standardized interfaces: an FHIR server’s RESTful interface. However, the results of this benchmark show that frameworks purpose-built for analytics can execute common queries significantly faster. A key factor in this performance gap lies in the design differences between online transaction processing and online analytical processing systems. FHIR servers, typically designed for transactional workloads or accessing individual patient records, are not optimized for computationally intensive operations such as aggregations or large-scale data filtering and extraction. In contrast, Trino and Pathling leverage modern distributed data processing paradigms. These systems enable efficient execution plans by integrating data retrieval, computation, and caching mechanisms directly within their frameworks. Still, for the count workload, we found Blaze to outperform the tested alternatives for simpler queries and lower record counts.

For the aggregation and extraction scenario, one performance-limiting factor was the need to fetch the results from the FHIR server sequentially using pagination, as the FHIR Search specification does not define native aggregation operations. The FHIR-PYrate implementation, therefore, was required to retrieve all matching “Observation” resources and perform grouping client side in pandas. In contrast, both Trino and Pathling executed the aggregation directly within the query engine using optimized query planning. The FHIR-PYrate library does allow for parallel fetching of search results, but this requires specifying an expected beginning and end date based on which to partition the search queries. For these scenarios, using the FHIR bulk export standard may be a higher-performing alternative [28].

We limited the selection of queries to only those that could be implemented as a single FHIR Search query. This avoids adding too much postprocessing logic to the total execution time. However, this also revealed challenges in the FHIR Search expressiveness and implementation in the server [5]. For example, the diabetes query in the count scenario was initially supposed to count the number of distinct patients satisfying the criteria. This would require a count on the “Patient” resource type filtered using reverse chaining (“_has”) to only include those patients who were referenced in a diabetes “Condition” and in an “Encounter” in the specific date range within which the diagnosis was recorded. However, multiple “_has” parameters are processed independently, which would cause “Patient” resources with diabetes “Conditions” recorded as part of any “Encounter” to be counted as well. Furthermore, FHIR Search requires explicitly defining search parameters for nonstandard resource elements (such as for searching within extensions). Depending on the implementation, this also requires reindexing all resources before searches can be executed using custom parameters. Even the availability of standard search parameters varies between servers. Compared to FHIR Search, Trino is not as limiting in its expressiveness, as it is an American National Standards Institute SQL-compliant query engine. Similarly, Pathling is based on Apache Spark, where both SQL queries and a DataFrame API can be used to query the data.

Both Pathling and Trino access the same data source—FHIR resources encoded as Delta Lake tables—but use different implementations to query them. This is an example of storage-compute decoupling, where storage and computation engines can be scaled and replaced independently. This also allows for querying the data using any other framework that supports reading Delta Lake tables and makes it easier for users to use tools they are already familiar with. Despite these advantages, the transition from FHIR-native data structures to tabular formats introduces challenges: encoding FHIR resources in Delta Lake tables requires additional infrastructure and processing steps. This process not only incurs extract, transform, and load overhead but also necessitates standardization efforts as the schema of these tables is not standardized and currently bound to the implementation of the Pathling encoders. Standardizing the encoding of FHIR resources as “Open” table formats might, therefore, be a useful future endeavor to ensure interoperability for analytics [29]. Systematic performance benchmarking of alternative encodings of FHIR resources would be valuable future work as schema design choices can affect query performance across analytics engines.

The resource use measured when loading the Synthea resources from the disk to the FHIR servers and Delta Lake tables differed across the tested system. Using Pathling, the FHIR JSON files were encoded faster and ended up using less disk space than both the HAPI and Blaze FHIR servers. However, Pathling also used more CPU and memory during loading. The 2 FHIR servers were loaded using 1 transaction bundle per Synthea patient record, and even though the blazectl client concurrency was set to 32, the servers did not fully saturate the available CPUs. This could be a result of the transaction processing implementation limiting the maximum concurrency. In comparison, the Pathling server used bulk import functionality, which was likely less affected by database locks and similar bottlenecks necessary to ensure atomic transaction processing. If it were implemented in all evaluated systems, consistently using bulk import would make the results more comparable.

The limited cache sensitivity observed for count workloads suggests that these queries were dominated by full-table scans rather than metadata or plan reuse. In contrast, extraction workloads benefited from cache reuse in Pathling and Trino, likely reflecting reuse of Parquet or Delta Lake metadata, execution plans, and file system caches in Spark and MinIO. While Trino caches Delta Lake metadata by default (for 30 minutes), we did not enable file system caching, which might have shown a more significant impact on cache-warmed performance.

The HAPI FHIR server was unable to complete the hemoglobin query beginning with the lowest record counts. When attempting to run the count variant of the query, we observed a single PostgreSQL CPU max out until eventually the query was aborted with an “out of disk space” error. We believe this may be caused by the way in which multiple OR-combined code-value-quantity searches are translated to SQL queries in the server.

The observed differences in the impact of data skew on the count performance highlight how engines respond differently to skew in predicate selectivity and result cardinality. Pathling’s and Trino’s insensitivity to code frequency suggests that performance is dominated by full-table scans, with limited benefit from selective predicates in the current execution model. In contrast, Blaze benefited substantially from highly selective predicates. Mixed-code queries remained dominated by high-frequency codes, explaining why their performance resembled hot-code workloads. These results demonstrate that realistic long-tail code distributions can affect engine performance and that benchmarks assuming uniform code distributions may overstate the performance of scan-dominated pipelines. The present experiment isolated skew in single-table predicate selectivity; join-key skew and shuffle skew in multi-table and multi-worker cohort queries remain important directions for future work.

All benchmarks were run on a single virtual machine, which means that both the benchmarked components and the code used to observe them were not isolated from each other. Additionally, all network communication occurred on the same local loop-back network interface, making the results less realistic as the impact of network latency was not measured. As modern data processing frameworks are capable of—and even optimized for—running analytical queries distributed across multiple compute nodes, how these findings generalize to such setups necessitates further exploration.

Future research should aim to establish a common set of benchmark scenarios tailored to health care analytics on FHIR. Drawing inspiration from benchmarks such as TPC-H [30], such a framework could provide a standardized methodology for evaluating diverse analytics engines and facilitate comparison across tools and architectures. This is particularly relevant with the new SQL-on-FHIR standard [7].

While our results show that FHIR Search performs poorly for analytical workloads dominated by large scans and aggregations, FHIR servers in general provide strengths that are critical in production environments: fine-grained access control; provenance and audit logging; transactional guarantees; and standardized functionality for searching, creating, updating, patching, and deletion. In many deployments, these features outweigh analytical throughput and justify the use of FHIR Search for patient-level queries and standardizable workflows [31]. Our findings, therefore, do not argue against FHIR servers as a primary clinical data interface but, rather, highlight their unsuitability as a sole back end for large-scale cohort analytics and population-level queries, where columnar analytics engines can provide substantial performance advantages.

The FHIR standard is primarily designed for interoperability and the exchange of health care data. In particular, while the REST-based FHIR Search API is useful for standard queries and retrieving specific patient records, it is generally not suitable for high-performance, complex analytics, making it necessary to move the data to a more robust analytical environment. Engines such as Trino and Pathling demonstrated substantial performance advantages for analytics scenarios.