This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.
Health information exchange (HIE), the electronic sharing of clinical information across the boundaries of health care organizations, has been promoted to improve the efficiency, cost-effectiveness, quality, and safety of health care delivery.
To systematically review the available research on HIE outcomes and analyze future research needs.
Data sources included citations from selected databases from January 1990 to February 2015. We included English-language studies of HIE in clinical or public health settings in any country. Data were extracted using dual review with adjudication of disagreements.
We identified 34 studies on outcomes of HIE. No studies reported on clinical outcomes (eg, mortality and morbidity) or identified harms. Low-quality evidence generally finds that HIE reduces duplicative laboratory and radiology testing, emergency department costs, hospital admissions (less so for readmissions), and improves public health reporting, ambulatory quality of care, and disability claims processing. Most clinicians attributed positive changes in care coordination, communication, and knowledge about patients to HIE.
Although the evidence supports benefits of HIE in reducing the use of specific resources and improving the quality of care, the full impact of HIE on clinical outcomes and potential harms are inadequately studied. Future studies must address comprehensive questions, use more rigorous designs, and employ a standard for describing types of HIE.
PROSPERO Registry No CRD42014013285; http://www.crd.york.ac.uk/PROSPERO/ display_record.asp?ID=CRD42014013285 (Archived by WebCite at http://www.webcitation.org/6dZhqDM8t).
In recent years, there has been substantial growth in the adoption of the electronic health record (EHR) in ambulatory and hospital settings across the United States, fueled largely by incentive funding provided by the Health Information Technology for Economic and Clinical Health (HITECH) Act. Following HITECH, 94% of nonfederal hospitals [
One key challenge to effective use of HIT, however, is that most patients in the United States, especially those with multiple conditions, receive care across a number of settings [
The Office of the National Coordinator for Health Information Technology (ONC) has defined the following forms of HIE [
ONC also uses the words “push” to describe directed exchange and “pull” to describe query-based exchange [
The HITECH Act recognized that EHR adoption alone was insufficient to realize the full promise of HIT, allocating US $563 million for states or state-designated entities to establish HIE capability among health care providers and hospitals [
Evaluating the effectiveness of HIE (and HIT generally) has been challenging [
There are 3 previously published systematic reviews that focus exclusively on HIE [
Key questions guiding this review were developed by the review team with input from a group of stakeholders and the Agency for Healthcare Research and Quality (AHRQ). A standard protocol was developed using input from key informants and a technical expert panel, registered in PROSPERO [
Details of included studies were extracted by one investigator and reviewed for accuracy and completeness by a second investigator. Two investigators independently assessed risk of bias for all effectiveness studies. Differences were resolved by discussion and consensus and reviewed by the team of investigators. Individual studies were rated as “low,” “moderate,” or “high” risk of bias. Investigators then assessed the strength of the body of evidence. Both the risk of bias and strength of evidence ratings were conducted using the criteria and procedures described in the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews [
The strength of evidence consisted of the following 4 major categories: high, moderate, low, or insufficient, based on the methodological limitations of studies; consistency across studies; precision of estimates; and directness of effect. Ratings were reviewed by a second investigator, and disagreements were resolved by consensus or involvement of a third investigator if necessary. Data could not be combined in a quantitative meta-analysis because of heterogeneity in the interventions, the outcomes measured, and the way data were reported. Therefore, we combined studies qualitatively based on the similarity of the type of HIE, the implementation of the HIE, outcomes measured, and results reported. Where studies were not similar in these areas, we provided the results of the individual studies without grouping them.
Of the 5211 potentially relevant citations identified in our literature searches, 849 articles were selected for full-text review and 34 studies were ultimately deemed to address outcomes. Study characteristics, results, and risk of bias assessments are presented in
Of 34 studies, 26 reported clinical, economic, or population outcomes (see
The most common study design for assessing outcomes was retrospective cohort, typically with HIE use associated with a specific outcome (
The identified studies were performed mostly in the United States, but we identified 8 studies from 5 other countries. Of the 26 studies in the United States, 2 assessed multiple HIE implementations across the entire United States, 1 assessed multiple HIE implementations in 2 states (California and Florida), and the remaining 23 studies were conducted in 13 states. Most studies used retrospective designs, usually with an approach examining the association of HIE use with 1 or more clinical variables. All of these studies focused on the direct effect of HIE, usually reporting reduction in resource use or costs, without determining its larger impact (eg, overall total or proportion of spending in an ED vs the total dollars that HIE appeared to save). None of the studies analyzed individual episodes of care to determine clinical appropriateness of possible changes brought about by HIE use.
The prospective studies also had limitations. The 2 RCTs (reported in 3 papers) were focused on highly specific uses of HIE, namely, directed exchange of ED reports in one and pharmacotherapy clinical decision support in another. Of note, however, was that neither study showed benefit of HIE. The other prospective study was a case series that was limited by its methodology relying on physician self-reports of resources not utilized when HIE was used, with no follow-up or validation of their decisions, or analysis of more holistic views of clinical outcomes or costs.
Study designs and locations.
|
Study designs and locations | References | |
|
|
|
|
|
Retrospective cohort (18) |
[ |
|
|
Survey (8) |
[ |
|
|
Randomized controlled trial (2 reported in 3 papers) |
[ |
|
|
Cross sectional (2) |
[ |
|
|
Case series (2) |
[ |
|
|
|
|
|
|
Austria (1) |
[ |
|
|
Canada (2) |
[ |
|
|
Finland (2) |
[ |
|
|
Israel (2) |
[ |
|
|
South Korea (1) |
[ |
|
|
All of United States (2) |
[ |
|
|
California and Florida |
[ |
|
|
Colorado (1) |
[ |
|
|
Indiana (3) |
[ |
|
|
Louisiana (1) |
[ |
|
|
Massachusetts (1) |
[ |
|
|
Minnesota (1) |
[ |
|
|
North Carolina (1) |
[ |
|
|
New York (6) |
[ |
|
|
Oklahoma (1) |
[ |
|
|
South Carolina (1) |
[ |
|
|
Tennessee (3) |
[ |
|
|
Texas (1) |
[ |
|
|
Virginia (1) |
[ |
|
|
Wisconsin (2) |
[ |
Most of these studies had reasonable but not strong internal validity. As the intervention (HIE) was only one of many potential influences on clinical outcome (ie, many more factors go into clinical outcomes than the decision to consult an HIE on a patient), there was possible confounding. Because no confounders were explicitly identified and incorporated into the analyses, most studies with appropriate retrospective methods were rated as having low or moderate risk of bias.
Because of the type of study designs used, reporting limitations, and the lack of ability to combine results, the strength of this body of evidence was rated as low, meaning that future studies have the potential to alter these findings in magnitude or direction. In addition, the number of studies and their locations in the United States represent a small fraction of functioning HIE systems. A larger number are reported to be operational, sustainable, or innovating according to the eHealth Initiative Annual Data Exchange Survey, which reported a total of 84 such HIE implementations in 2013 [
Most of the studies of HIE effectiveness focused on resource use. We categorized these as follows (
Study results by categories.
Category (number) | Results |
Laboratory testing (6) |
A total of 6 studies showed benefit for health information exchange (HIE) in reducing overall testing, although estimates of impact on cost were mixed [ |
Radiology testing (9) |
A total of 7 studies carried out in the ED setting showing reduced testing [ |
Hospital admissions (8) |
A total of 2 studies found a reduction in hospital admissions and lower costs [ |
Hospital readmissions (2) |
Whereas 1 study showed benefit for HIE in reducing hospital readmissions [ |
Referrals and consultations (2) |
A total of 2 studies assessed HIE for reducing referrals and/or consultations, with conflicting results [ |
ED costs (2) |
A total of 2 studies found reduced overall ED costs per patient when HIE was available [ |
Public heath reporting (3) |
A total of 3 studies assessed HIE in public health settings, all of which were conducted in the United States and reported improved automated laboratory reporting [ |
Quality of care in ambulatory settings (3) |
A total of 2 retrospective studies found HIE associated with improved quality of care [ |
Other aspects of HIE (3) |
A total of 3 studies assessed other aspects of HIE, including reduction in time for processing of Social Security Disability claims [ |
A number of studies evaluated clinician or patient perceptions of outcomes of HIE (see
To determine whether effectiveness of HIE varied by study type, health care setting, location, or HIE type, we examined whether HIE was found to have some beneficial effect or not across characteristics. As presented in
A collection of low-quality evidence supports the value of HIE for reducing duplicative laboratory and radiology test ordering, lowering ED costs, reducing hospital admissions (less so for readmissions), improving public health reporting, increasing ambulatory quality of care, and improving disability claims processing. The evidence is low quality because of the retrospective nature of the studies and the limited questions that they ask. It is unlikely that additional studies of the kind included in this review will advance the field and strengthen our understanding when HIE can reduce laboratory and imaging tests associated with episodes of care without broadening their scope and using more rigorous designs. Although the preponderance of evidence reports positive effects of HIE in reducing resource use and improving quality of care, it is entirely possible that focused studies with stronger study designs and more comprehensive assessment of utilization or clinical outcomes might reach a different conclusion.
We found no studies explicitly addressing patient-specific clinical outcomes such as morbidity, mortality, or functional status, and therefore the body of evidence is insufficient to determine whether HIE has an impact on patient outcomes. We also did not identify any studies that used systematic and comprehensive economic analysis. Although some of the studies we included projected or estimated cost savings based on measured changes in utilization or perceptions of clinicians, there were no studies that explicitly measured costs and assessed economic impact in a comprehensive fashion. It is fair to say, then, that there was insufficient evidence to reach conclusions on the economic impact of HIE.
How likely are the effects reported in this review to be observed when applied under diverse conditions in health systems, hospitals, and clinics in the United States? The greatest confidence in the applicability of these findings comes from the breadth of settings—geographic, organizational, and technical—from which they are derived. By contrast, there are limitations to the applicability of the findings (beyond limitations to the internal validity already mentioned) having to do with these main concerns: (1) concentration of evidence from a relatively small number of HIE systems; (2) use of internally developed and refined health IT systems compared with local instances of commercial systems; and (3) the exceptionally broad variety of systems, contexts, and purposes of HIE reported in the studies included in this review.
First, the concern that the bulk of the evidence about health IT impact arises out of a relatively small number of centers has been raised before [
Second, separate from the “health IT leader” concern, which has to do with the organizational capacity, resources, and mission of these centers, is the issue of internally developed systems compared with commercially developed systems. Although few of the studies we included described whether their software used was commercial or locally developed, the overall model of health IT purchase and installation of nonhealth IT leaders are usually quite different from that of the incremental internal development, implementation, and refinement that are seen in systems such as the Department of Veterans Affairs or the aforementioned “health IT leader” systems. Related to this concern is a finding from other aspects of health IT [
Third, and most important, in terms of limiting the applicability of these findings about HIE to real-world use is the exceptionally wide variety of systems, purposes, and contexts of use. To predict whether specific implementations of HIE in specific health care contexts will have favorable impacts on specific desired outcomes is not possible from this review and in most cases would not be possible from comparison with individual studies because (1) it is unlikely that studies with low risk of bias have been published for most such specific questions, and (2) in almost all cases these are complex interventions that are incompletely specified, with insufficient detail to draw strong meaningful inferences [
The significant limitations of the evidence base, that is, the individual studies included in this review, have been raised in previous systematic reviews of health IT [
First, the evidence in this area addresses a wide variety of questions covering diverse domains beyond medical science from computer science, human factors, sociology, organization and management, and other disciplines. This broad array of questions calls for an equally diverse range of study designs. Studies of usability and use require usability engineering methods, studies of individual behavior call for methods from anthropology and behavioral sciences, studies of organizational change warrant methods drawn from management and systems science, whereas studies of population effects call for the methods of epidemiologists. A significant limitation of this literature, with its breadth of research questions, is the limited toolbox often drawn upon to answer them.
The second limitation is in execution of the studies. Even when strong study designs are used, their execution may be lacking, whether in sampling strategies, measurement methods, or analytic approaches. The unit of analysis problem is but one example. Interventions carried out at the level of the health system, hospital, or clinic may be analyzed at the level of the patient or episode, without controlling for variation at these multiple levels. Incomplete measurement is another: for example, where ED test ordering is measured in isolation, ignoring the possibility that the same test might later be ordered in another setting such as urgent care, primary care, or in hospital.
The third limitation has to do with the complexity of interventions, where the HIE or other health IT system itself is necessarily only part of a more complex intervention. The complexity of interventions to change the behavior of clinicians or others in the health systems studied requires more thorough specification, to both adjust for confounders and make sense out of how to apply interventions elsewhere. Others have documented the inadequacy of specification of the details of complex interventions and called for a more systematic and thorough reporting [
Finally, the literature does not comprehensively cover changes in technology or policies governing its use. For example, whereas most studies come from the locally developed systems of HIE leaders as noted earlier, there has been a more recent growth in the commercial marketplace for HIE. In addition, the widespread adoption of EHRs under the HITECH Act in the US means that a more diverse array of health care organizations will be participating in HIE implementations. As an example of policy changes governing HIE development, as noted in
Given the limited conclusions that can be reached after review of the large volume of published literature on HIE, what are the implications for future research? Recognizing that HIE, like health IT in general, will almost certainly undergo increasingly widespread implementation in the future, the first aim of researchers should be to shift the emphasis from
A second aim of research on HIE should be to develop greater focus and clarity about the level at which interventions are operating and the types and levels at which outcomes are measured. The outcomes of interest and the factors influencing them may be quite different at different levels of analysis, from specific systems or functionalities of HIE to individual patients, providers, or episodes of care; to health care units such as the ED, primary care practice, or hospital ward; to institutions such as hospitals; to aggregates such as health systems; or to broader regional multiorganization entities or regions. Combining or confusing these levels of intervention and levels of analysis only increases the challenges for those who conduct the research and for those who wish to interpret and apply it.
To help achieve an improved focus and clarity, a more formal analytic framework and a more descriptive taxonomy are needed. An example of such a framework that could be usefully applied in this area is Rasmussen’s sociotechnical hierarchy, which specifies the multiple levels of a complex sociotechnical system that must be considered together to understand system behavior change [
Similarly, a formal taxonomy for implementation of complex interventions has been proposed that would enable more complete and useful specification of interventions to allow better analysis, interpretation, and application [
The third step researchers can take to improve the evidence base for implementation of HIE is to broaden the methodologic toolbox applied to these questions. As indicated earlier, the study approach and architecture must be suited to the question being asked, employing methods from usability engineering, behavioral sciences, systems engineering, and organizational sciences, depending on the question being addressed. These would include methods used in engineering and quality improvement, as well as in the study of complex adaptive systems.
What types of studies should be performed? RCTs are impractical for technologies with wide-ranging purposes like HIE. Yet, retrospective studies associating HIE versus nonuse for outcomes such as test ordering and hospital admissions are very limited in conclusions that can be drawn. Research is also challenging because many of the important clinical outcomes that could be positively affected by HIE have many other potential contributing and confounding factors relating to the patient, his or her clinicians, the quality of care delivered, the EHR, other health IT used, the nature of the health care delivery system, and the regulatory environment. Given the growing evidence based on robust evaluations in other areas of health IT, as noted in systematic reviews [
Future studies should be prospective, carried out in mature HIE settings, specify a priori what patients and/or use cases are likely to benefit from HIE, and compare appropriate outcomes for the use or nonuse of HIE. The prospective collection of data from diverse settings where HIE is used, classified by the taxonomy advocated earlier, could allow for prospective cohort studies that could identify aspects of HIE associated with beneficial outcomes. This will likely require an effort comparable in scope to national data collection efforts, such as the Patient-Centered Outcomes Research Institute Clinical Data Research Network initiative [
Evaluation should be a requirement for all HIE implementations, certainly those funded by grants or other external funding. The challenge of evaluating health IT projects, especially in community settings, is well-known [
The full impact of HIE on clinical outcomes and potential harms is insufficiently studied, although evidence provides some support for benefit in reducing use of some specific resources and achieving improvements in quality of care measures. To advance our understanding of HIE, future studies need to address comprehensive questions, use more rigorous designs, and be part of a coordinated, systematic approach to studying HIE. Going forward, HIE will become a more integrated part of health care delivery, and its evaluation needs to be focused on maximizing the improvements that HIE usage brings to overall clinical care.
Studies of health information exchange included for assessing outcomes.
Patient and clinician survey perceptions of health information exchange.
Agency for Healthcare Research and Quality
emergency department
electronic health record
health information exchange
health information technology
Health Information Technology for Economic and Clinical Health
information technology
Office of the National Coordinator for Health Information Technology
randomized controlled trial
The authors gratefully acknowledge the following individuals for their contributions to this project: Andrew Hamilton, MLS, MS, for conducting literature searches and Spencer Dandy, BS, for assistance with preparing this manuscript (both are located at the Oregon Health & Science University); and Jon White, MD, Task Order Officer at the Agency for Healthcare Research and Quality (AHRQ). The primary funding source was The Agency for Healthcare Research and Quality (Contract Number 290-2012-00014-I, Task Order 11), Rockville, Maryland. SZK was also supported by a grant from the National Library of Medicine Training (Grant No T15LM007088). The findings and conclusions in this document are those of the authors, who are responsible for its content, and do not necessarily represent the views of AHRQ or the National Library of Medicine. No statement in this report should be construed as an official position of AHRQ or the US Department of Health and Human Services.
None declared.