This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.
With the aim of improving health care processes through health information technology (HIT), the US government has promulgated requirements for “meaningful use” (MU) of electronic health records (EHRs) as a condition for providers receiving financial incentives for the adoption and use of these systems. Considerable uncertainty remains about the impact of these requirements on the effective application of EHR systems.
The Agency for Healthcare Research and Quality (AHRQ)-sponsored Centers for Education and Research in Therapeutics (CERTs) critically examined the impact of the MU policy relating to the use of medications and jointly developed recommendations to help inform future HIT policy.
We gathered perspectives from a wide range of stakeholders (N=35) who had experience with MU requirements, including academicians, practitioners, and policy makers from different health care organizations including and beyond the CERTs. Specific issues and recommendations were discussed and agreed on as a group.
Stakeholders’ knowledge and experiences from implementing MU requirements fell into 6 domains: (1) accuracy of medication lists and medication reconciliation, (2) problem list accuracy and the shift in HIT priorities, (3) accuracy of allergy lists and allergy-related standards development, (4) support of safer and effective prescribing for children, (5) considerations for rural communities, and (6) general issues with achieving MU. Standards are needed to better facilitate the exchange of data elements between health care settings. Several organizations felt that their preoccupation with fulfilling MU requirements stifled innovation. Greater emphasis should be placed on local HIT configurations that better address population health care needs.
Although MU has stimulated adoption of EHRs, its effects on quality and safety remain uncertain. Stakeholders felt that MU requirements should be more flexible and recognize that integrated models may achieve information-sharing goals in alternate ways. Future certification rules and requirements should enhance EHR functionalities critical for safer prescribing of medications in children.
The Health Information Technology for Economic and Clinical Health (HITECH) Act was signed into law on February 17, 2009 with the commitment of substantial financial resources to expand the use of electronic health records (EHRs) and great hopes of promoting improvements in the efficiency of health care for all Americans. This effort is being led by the Centers for Medicare & Medicaid Services (CMS) and the Office of the National Coordinator for Health Information Technology (ONC) at the US Department of Health and Human Services (HHS) [
The CMS EHR incentive programs have included 3 stages to date, each with its own specific objectives, measures, and standards. The final rules for MU Stage 1, which specify the criteria that eligible professionals and hospitals need to meet to qualify for incentives, went into effect on September 26, 2010. The rules defined 15 core and 10 menu-set objectives that focused on providers capturing and sharing patient data. The onset of MU Stage 2 criteria was delayed until 2014 and concentrated on advanced clinical processes and more rigorous health information exchange (HIE). Specific to the Stage 2 objectives was the expectation that patients would be provided with secure online access to their health information. The Health Information Technology Policy Committee (HITPC), which advises the government on its EHR incentive program, submitted its preliminary recommendations for MU Stage 3 to the ONC in early 2013. As part of the federal rule-making process, these preliminary Stage 3 recommendations were released for public comment and generated a high volume of responses [
We gathered the perspectives of a wide range of professionals (N=35) representing academicians, practitioners, policy makers, and senior management officials identified through the CERTs, henceforth referred to as “stakeholders.” Stakeholders initially met in June 2014 as part of the national CERT steering committee meeting to discuss the purpose and content of this document and included representatives from different health care and academic organizations including: Agency for Healthcare Research and Quality (AHRQ) (n=5), Kaiser Permanente (n=4), Brigham and Women’s Hospital (n=3), Cincinnati Children’s Hospital Medical Center (n=2), Food and Drug Administration (FDA) (n=2), Duke University (n=3), Rutgers University (n=2), University of Alabama at Birmingham (n=2), Intermountain Healthcare (n=1), University of Illinois at Chicago (n=1), Northwestern University (n=1), University of Washington (n=1), University of Maryland (n=1), Baylor Scott and White Health (n=1), Baylor College of Medicine (n=1), Blue Cross Blue Shield Association (n=1), and a variety of others (n=4). A number of stakeholders occupied roles such as Chief Medical Information Officer or Chief Medical Informatics Officer in their respective health care organizations. A number of open-ended questions were posed to the group including:
What were your experiences of implementing Stage 2 MU requirements?
What key challenges did you face?
How were these challenges overcome (or could they be overcome in the future)?
What are your thoughts on the proposed Stage 3 recommendations?
Do you think there were any important areas omitted in the proposed Stage 3 recommendations?
Specific issues and recommendations were presented, discussed, and agreed on as a group. Some of these issues that were agreed on by the stakeholders have been documented and supported by relevant literature. We used the principles of consensus decision making; all stakeholders were (1) involved in the group discussions (inclusive), (2) encouraged to contribute opinions and suggestions (participatory), (3) given the opportunity to build on one another’s suggestions (collaborative), (4) afforded equal input into the process (egalitarian), and (5) allowed to voice any particular concerns that they may have so that the group could incorporate them into the emerging domains (cooperative). These include, for example, how organizational differences in the delivery of health care could impact stakeholders’ ability to achieve MU requirements, challenges and opportunities for rural communities, and how EHRs could be improved to support safer and more effective prescribing for children. The public commentary available on the government website was reviewed to help inform these discussions [
When a patient is transferred from one health care setting or provider to another, it is essential that accurate and up-to-date information about the patient’s medications be provided. This enables health care professionals responsible for the patient’s care to identify any medication changes or discrepancies between the prior and current medication lists. This process of medication reconciliation helps health care providers make informed decisions and safely monitor their patients’ care [
Better electronic tools are needed to assist with this medication reconciliation process [
A Stage 3 recommendation was that EHR systems should provide functionality to help maintain an up-to-date accurate medication list (SGRP 106); therefore, the incorporation of external data, such as pharmacy dispense status notifications, into vendor EHR systems was proposed for a future stage of MU (SGRP 125). These data could better inform users as to whether a patient had their prescription(s) filled, was taking 2 kinds of the same drug (including detection of abuse), or was using multiple drugs whose indications overlap. All stakeholders agreed that such needed interoperability poses additional challenges related to data validity, reliability, and integrity, and concerns about the willingness, timing, and ability of pharmacies to make these data available electronically.
One specific recommendation from the stakeholder group was that medication cancelations should be transmitted to pharmacies. This is often done in the inpatient setting, but it is not done in the outpatient setting, although a standard does exist. If this were done, it could help resolve many discrepancies in medication reconciliation.
An accurate list of a patient’s problems and allergies represents a key component of the patient’s EHR. Problem lists contain a list of patients’ problems or diagnoses and may be used by clinicians to familiarize themselves with the needs of a patient and orient caregivers to the reasons why a patient may be on a particular medication or regimen. If a problem is properly documented in a patient’s EHR, their clinician can receive appropriate alerts and reminders to guide care. The problem list also helps primary care practices to correctly identify disease-specific populations and create patient registries ensuring that all patients benefit from the most up-to-date evidence-based care.
The MU Stage 3 recommendations expanded the scope of reconciliations to include those of medication allergies and problems (SGRP 302). Many stakeholders recognized the importance of obtaining patients’ input on the accuracy of problem lists (SGRP 105) in the process of reconciliation. However, concerns were raised by the stakeholder group about whether and how patients should contribute to the same up-to-date problem list as clinicians and, if so, whether this may confuse and possibly interfere with the credibility of the list [
One Stage 2 core measure recommended maintaining an up-to-date problem list of current and active diagnoses (SGRP 105) and a medication allergy list (SGRP 107). Stage 3 recommendations expand on these basic requirements proposing that EHR systems should also provide functionality to help keep both problem and allergy lists accurate and up-to-date. One stakeholder explained how the University of Washington has developed new functionality using natural language processing to help achieve this objective for EHR problem lists. However, because of the burden of complying with MU requirements, other work that was not directly tied to MU incentives was postponed or halted. For example, before the launch of the MU incentive program, there were active clinical decision support (CDS) initiatives on-going at the University of Washington for the early detection of sepsis, identifying non-ICU patients at risk of clinical deterioration, complying with guidelines to reduce ventilator-associated pneumonia, venous thromboembolism, and other complications of ICU care—all leading causes of patient harm. However, to meet MU requirements, work on these projects was deferred and the clinical analysts, engineers, and senior programming staff were redirected to work on implementing MU requirements. One stakeholder reported a similar stifling of innovation at Intermountain Healthcare, where the implementation of MU capabilities delayed other EHR development projects, such as the replacement of legacy system functionality in labor and delivery, electronic consent handling, clinical HIE workflow integration, and replacement/enhancement of inpatient computerized provider order entry (CPOE) functionality. The consensus of the stakeholder group was that this might represent an opportunity cost for innovation. Institutions understandably may place priority on innovations that will bring known rewards, even if the innovations would not be as high a priority if there were no incentives. These unintended consequences of the MU incentives can be instructive to consider as other “pay for performance” programs are initiated.
The definition of CPOE by CMS is “a provider’s use of computer assistance to directly enter medical orders (eg, medications) from a computer or mobile device” [
Stage 2 and 3 recommendations require EHRs to maintain active medication allergy lists (SGRP 107). This includes functionality that codes medication allergies and links them to related drug family and code-related reactions. It known that an allergy that is entered as free text in the EHR is neither interoperable across clinical information systems nor easily usable for CDS applications, such as drug-allergy interaction checking. However, the US government has not yet specified which standard terminologies should be used to structure and encode allergy information. The consensus of the stakeholder group was that defining allergy standards will be essential to facilitate both documentation and the exchange of information between health care settings [
Stakeholders, especially those from the Cincinnati CERT that specializes in pediatric medication use, expressed concern about the lack of attention paid to pediatric prescribing in the MU criteria. Although Stage 2 and Stage 3 MU objectives required CPOE systems to be used for 60% of medication orders (SGRP 101), EHR functionalities to assist with the prescribing of medications for children have not been specifically mentioned or recommended. This is despite the fact that prescribing medicines for children is reported in the literature to carry disproportionately higher safety risks and be more error prone compared to prescribing for adults [
The American Academy of Pediatrics (AAP), AHRQ, and Health Level 7 (HL7) International, have described desirable functionalities for EHRs in pediatric populations. Major areas include immunization management, growth tracking, medication dosing, data norms, and privacy in special pediatric populations [
Weight-based/body surface-based dose calculations and range checks [
Ability to detect erroneously entered weights [
Display of patient specific units of measure (eg, grams) along with the data values [
Rounding of medication doses to appropriate decimal precision with special consideration of the low-weight patients [
Display of data that influenced the final dose and amount in the prescription, particularly to dispensing pharmacists [
Display of normal pediatric dose ranges and advice when no pediatric references exist [
Use pediatric dose ranges for alerts using patient weight/age with soft-stops for adult dose [
Appropriate alerts for age correction for preterm infants, neonates, and low-weight patients [
Recommendation of optimized dispensing format (liquid, tablet, etc) or concentration for the patient [
Adolescent patients require a level of confidential care, especially when prescribing medications for reproductive or mental health issues [
Stage 3 recommendations propose a new measure that would require health care providers to generate and transmit discharge prescriptions electronically (SGRP 103). Although this objective may improve workflow for pediatric providers and reduce the risk of illegible handwriting and transcription errors, the stakeholder group felt that it does not focus on the decision support required to generate correct prescriptions and may simply enable faster generation and transmission of potentially erroneous orders. Current formats for electronic prescription messages do not include body weight or any details about the calculations that yielded the dose [
Awards totaling US $10 million were collectively granted to 5 domestic institutions to support HIT curriculum development in April 2010 to the University of Alabama at Birmingham, Johns Hopkins University, Columbia University, Duke University, and Oregon Health and Science University. Each of these Curriculum Development Centers was given responsibility to develop, revise, and share curriculum components covering a specific set of HIT content areas. The ultimate aim was to prepare future professionals to meet emerging workforce needs. Despite the initial HITECH funding for training, stakeholders felt that the needs of the HIT workforce in rural areas across the country have not been met yet. Rural communities are more likely to have smaller practices, which have been among the last to embrace electronic medical records [
According to the stakeholders, especially those from the University of Alabama at Birmingham CERT that specializes in workforce training, several steps have been taken to address these issues. In addition to the workforce training programs, 62 Regional Extension Centers (RECs) have been established with US $677 million in funding from the ONC to provide on-the-ground assistance to smaller rural practices. In 2011, the ONC announced an additional US $12 million in new technical support assistance to help CAHs and rural hospitals adopt and become meaningful users of certified HIT. This funding was in addition to the $20 million provided to RECs in September 2010 to provide technical assistance to the CAHs and rural hospitals [
One stakeholder pointed out that as more hospitals and practices begin to meet the MU criteria, some of the traditional boundaries that have separated rural primary care practices from tertiary care centers in large urban areas may begin to disappear. Primary care practices may have more access to information about their patients’ hospitals stays. Tertiary care hospitals are likely to have a substantial number of patients from surrounding rural areas who can benefit from patient portals or similar mechanisms to promote patient engagement (SGRP 204A). However, patient engagement is likely to be another challenge going forward with rural residents, considering unreliable Internet connections, low health literacy, and lack of resources. Although MU requirements currently set a low percentage of patients who are expected to use the portals, the consensus of the stakeholder group was that systems must be scalable if more patients are to benefit, which will likely entail use of novel technologies such as mobile devices.
Many different stakeholders supported the MU general goal that providers should have appropriate information about patients transitioning into their care (SGRP 303). Stage 3 recommendations expanded on this Stage 2 objective by specifying the types of information that should be included in the summary care record, such as a concise narrative section, goals, instructions, and care team members. The consensus of the stakeholder group was that some organizations, such as Kaiser Permanente or Intermountain Healthcare, might find it easier to achieve this objective than others. Such well-established integrated delivery systems have organized, coordinated, and collaborative networks that bring together various health care providers to deliver coordinated care to a defined patient population [
Another issue raised by a different stakeholder related to whether organizations are using existing functionality (eg, Surescripts) or have chosen to develop their own. Kaiser Permanente and other integrated delivery systems lacked the functionality to bring medication information from external pharmacies into their EHR system and were swayed by the MU incentives to add this to their systems. However, the value of this functionality within staff model systems such as Kaiser Permanente is likely to be low in light of the fact that Kaiser Permanente patients obtain nearly all their medications from Kaiser Permanente. Stakeholders agreed in principle that external interoperability functionality can help maintain accurate medication and problem lists, although they felt that implementation should be flexibly based on the organizational-specific contexts. They also felt that many of the specific criteria should be postponed until the technological, operational, and legal issues are more fully evolved and the quality and accuracy of tools are sufficiently tested.
Finally, Stage 3 recommendations propose a new measure that would require health care providers to use CPOE for referrals/transition of care orders (SGRP 130). One stakeholder highlighted how some organizations, including Intermountain Healthcare, already use extensive CPOE/CDS capabilities and other advanced functionality and questioned the value of spending considerable resources to develop functionality that they believed would add little to their existing systems simply to meet MU requirements. For example, for the successful attestation of Stage 1, Intermountain Healthcare estimated that its 696 eligible professionals and 22 hospitals were eligible for approximately US $46.3 million. The high degree of coordination already inherent in their delivery model and IT systems meant that total costs for the implementation of Stage 1 recommendations were considerably lower than for others at an estimated US $17.3 million, resulting in a net revenue benefit of US $29 million. Although this financial benefit may seem substantial, another stakeholder pointed out how these total implementation costs may not reflect the “true” cost because they did not include the development of the system’s computer network (in their case, this was already in existence) or the disruption caused by HIT implementations and upgrades. Therefore, the consensus of the stakeholder group was that it is important to understand the current structural advantages of existing integrated delivery systems in the achievement of MU objectives and to recognize the need for future MU requirements to be applied and interpreted more flexibly.
1. Accurate Medication Lists and Medication Reconciliation
The quality and accuracy of these medication lists is often poor and providing patients with medication lists that are of dubious quality can pose a risk to patient safety.
Better electronic tools are needed to assist with this medication reconciliation process.
The incorporation of external data, such as pharmacy dispense status notifications, into vendor EHR systems could better inform providers about a patient’s medicines usage.
2. Accurate Problem Lists and the Shift in HIT Priorities
EHR systems should also provide functionality to help keep both problem and allergy lists accurate and up-to-date.
Institutions understandably may place priority on innovations that will bring known rewards, even if the innovations would not be as high a priority if there were no incentives.
All CDS alerts need to be implemented thoughtfully with careful attention paid to the balance between sensitivity and specificity, and how the alerts are delivered to providers.
3. Accurate Allergy Lists and Allergy-Related Standards Development
Defining allergy standards will be essential to facilitate both documentation and the exchange of information between health care settings.
4. Supporting Safer and More Effective Prescribing for Children
Data norms and values change continuously with age and EHRs can assist with the calculation and flagging of abnormal values.
Few Stage 2 requirements were aligned sufficiently with the functionalities considered critical for the accurate prescribing of medications in children and it was key that this issue be addressed in the development of future recommendations.
5. Challenges and Opportunities for Rural Communities
Despite the initial HITECH funding for training, the needs of the HIT workforce in rural areas across the country have not been met.
Patient engagement is likely to be challenge going forward with rural residents, considering unreliable Internet connections, low health literacy, and lack of resources.
Although MU requirements currently set a low percentage of patients who are expected to use the portals, systems must be scalable if more patients are to benefit, which will likely entail use of novel technologies such as mobile devices.
6. Achieving MU: Easier for Some Than for Others?
Some MU measures have been easily achievable for integrated delivery systems, whereas other measures have required substantial resources to fund and have impacted only a small portion of their patient population.
Future MU requirements need to be applied and interpreted more flexibly.
We assessed stakeholders’ learning and experiences from the implementation of MU requirements over the past 4 years, with a particular focus on medication requirements and attempted to identify problem areas where midcourse corrections might be helpful. Six specific issues were highlighted, all of which present opportunities for improvement. The implementation of MU capabilities was reported to have stifled innovation at some organizations. This appears to run counter to the ONC’s goal of encouraging innovation and creating “an environment of testing, learning, and improving, thereby fostering breakthroughs that quickly and radically transform health care” [
Likewise, although the EHR incentive program was viewed as a valuable opportunity to encourage provider-level clinical quality measure (CQM) innovation and perform provider-level CQM testing, some stakeholders felt it distracted them at least temporarily from their efforts to develop and implement such quality measurement and improvement systems. The HITPC also raised the possibility of allowing health care organizations to submit a locally developed CQM as a menu item, in lieu of one of the existing measures specified in the MU program [
Definitions of transitions in care should enable and support shared patient record systems. Better tools and interoperability with external data are needed for effective and efficient medication reconciliation. On the other hand, measures should not drive unnecessary or unreliable data transmission.
Development of a common terminology model is needed to facilitate documentation and encoding of key data elements, notably patient allergies.
Future MU certification rules and requirements should consider EHR functionalities that are critical, but often lacking, for the accurate prescribing of medications in children.
Future MU requirements should put more emphasis on flexibly understanding, incorporating, and supporting local HIT configurations that address population health needs.
The MU objectives should acknowledge the diversity of health care systems. For example, integrated delivery systems are more likely to achieve the goal of information sharing because of their integrated structure, greater functionality, and improved interoperability. From the policy perspective, this could be handled by offering exceptions or alternate routes for qualification.
The sampling strategy used in this study ensured that the perspectives of highly knowledgeable informants from the 5 AHRQ-sponsored CERTs were captured. Our sample included those directly involved in the implementation of MU criteria (eg, Chief Medical Information Officer or Chief Medical Informatics Officer) and those who were knowledgeable about, but not directly involved in, the day-to-day implementation work (eg, academicians, practitioners, policy makers). Participants were free to raise any issues that they felt were relevant to the topic under discussion. Consequently, we believe that the information gathered was reflective of genuine concerns and views. All stakeholders were given an opportunity to provide feedback on the key domains, ensuring that the conclusions accurately reflected the opinions and views collected. A limitation of this study was that it was performed in the US context and, therefore, could be viewed as less applicable to other countries. However, we believe that the implementation and adoption of EHRs is highly heterogeneous across health care systems and countries, and will be of interest.
The future course that the federal government will take with respect to HIT and policy measures is uncertain. It is not clear whether there will be a fourth stage of MU, although that currently seems unlikely. Taking stock of the important ways MU has been successful in achieving many of its objectives—such as dramatically increasing the number of medications ordered electronically—as well as where it encountered predicted and unanticipated problems, will be critical to mapping the next steps. Overall, the incentives and specific MU criteria will almost certainly be less important than they have been in the future as information systems more broadly improve their functionality and many of the challenges that we face today become embedded as the standard of care. It does appear that certification will continue to be important, although providers have recently called for separating MU from certification [
Regardless, we believe it will be important for the federal government to address some of the issues we have identified in this paper, including problems with how medication reconciliation is being promoted, the issues around accurate problem lists and the shift in HIT priorities, supporting safer and effective prescribing for children and rural communities, and making achieving MU more likely to result in the care improvement desired by all stakeholders. Any new policy will introduce new problems and it is essential for the federal government and others to consider how best to address these issues and others through the MU incentive program.
Critical Access Hospital
clinical decision support
Centers for Education and Research in Therapeutics
computerized provider order entry
clinical quality measure
drug-drug interaction
electronic health records
Food and Drug Administration Safety and Innovation Act
health information exchange
health information technology
Health Information Technology for Economic and Clinical Health
Health Information Technology Policy Committee
Health Services and Resource Administration
meaningful use
Regional Extension Centers
Unique Ingredient Identifier
We would like to thank the CERTs Scientific Forum for agreeing to host these meetings.
DWB conceived this manuscript. SPS reviewed the public commentary available on the government website and led the writing of this manuscript with TYW, SAS, CUL, NM, ESB, BG, BLL, THP, SH, and MM contributing. All coauthors commented on subsequent drafts and gave their approval for the final version to be published. SPS and DWB acts as guarantors.
Dr Bates and Dr Lehmann have served as members of the HIT Policy Committee. Dr Huff has served as a member of the HIT Standards Committee.