A search of MEDLINE/PubMed, CINAHL, PsycInfo, IEEE Xplore, and Web of Science was conducted using the keywords “clinical decision support,” “decision support acceptance,” and “user acceptance.” No timeframe limits were included for any database, and the language filters were set to English studies only. In our initial search, we found 186 papers. After removal of duplicates, 150 studies remained. To be included in this review, the papers had to match the following inclusionary criteria: investigate human interaction with a CDSS and evaluate user acceptance using the TAM questionnaire, focus groups, or interviews. Papers were excluded if the focus was on decision support systems that did not include clinical care or if they did not empirically investigate user acceptance. Title and abstract review eliminated 111 studies. The remaining 39 studies underwent a full-text review, resulting in a final count of 14 studies that met inclusion criteria. The search results are summarized in Figure 1.

Study findings were categorized as either showing favorable or unfavorable responses to CDSSs. The favorable and unfavorable categorization was based on interpretations of focus groups and interviews conducted by the researchers of the reviewed papers. Additionally, the type of CDSS was noted for each of the reviewed papers. If a study used the TAM questionnaire, the results were summarized separately.

To gain a greater understanding of the problems associated with CDSSs, we conducted a task analysis. Using past research, the task analysis helped identify and describe the goals, user input, system output, knowledge requirements, and constraints from two different perspectives. We considered the perspective of the machine (ie, the CDSS engine). We also considered the perspective of the user (ie, the physician). The literature review and task analysis served as the basis for designing CDSS models that improved user acceptance.

The results of 14 articles were evaluated. The 11 articles that qualitatively evaluated user acceptance of CDSSs can be found in Table 1 and the three articles that quantitatively evaluated user acceptance of CDSSs using TAM can be found in Table 2. Favorable and unfavorable responses for the aspects of clinical guidelines, reminders, and diagnostic CDSSs were recorded. Favorable responses were due to ease of system use, perceived time savings, and perceived usefulness of the systems in improving care delivery and overall patient health [14]. Users with higher computer skills were reported to have greater acceptance; however, the majority of users had an unfavorable acceptance response [15]. These unfavorable responses were often related to workflow interference, questionable validity of the systems, excessive disturbances caused by the systems, and lack of efficiency. More specifically, the workflow constraints were related to the CDSSs causing excessive alerts, increased time in computer handling, and decreased face-to-face time with patients [14,16,17].

Of the studies reviewed, three used the TAM questionnaire for assessing user acceptance of CDSSs (Table 2). The CDSSs in these studies included an evaluation of two different computerized clinical guideline systems and one that offered reminders and alerts for evidenced-based guidelines. The ranges on perceived usefulness all overlap, but on the CDSSs with the highest perceived usefulness it also has the highest perceived ease of use. Overall, the TAM questionnaire revealed moderate user acceptance on all scales. In terms of the relationship between user acceptance of CDSSs and patient safety, none of the reviewed papers evaluated this topic. However, Bergman and Fors [15] found that the use of the technology was relatively low when user acceptance was low.

Task analysis is conducted to stay updated with the changing professional practice (ie, health information technology) [29]. Task analysis applied to representative populations strengthen health systems by systematically evaluating the skills, knowledge, and behavior of clinicians that impact clinical practice [30]. The use of CDSS in health care has introduced new dynamics to practice and requires task analysis to understand the perception of users to this new technology. For that reason, conducting a task analysis will improve adoption levels. A task analysis includes goals, input, process, and output. The next sections discuss the purpose of each task analysis stage.

Physicians are more likely to accept a CDSS if the system matches their own decision-making processes. Forster [33] described how humans quickly act on information by using bounded and ecological rationality. Bounded rationality is based on the use of simple heuristics, allowing for fast, real-time decision making [34]. Ecological rationality is based on rational beliefs of things in a given environmental setting where conditions are fluid. Forster [33] argued that both bounded and ecological rationality need to be present in machine learning to mimic the human decision processes.

Incorporating these two approaches into CDSSs can be challenging. Even though the heuristics that mediate decision processes are simple; the complexity of the cognitive infrastructure underlying heuristic operations can be difficult to implement. Still, Forster [33] argued that machine learning algorithms can be improved by incorporating the principles of bounded and ecological rationality. To carry out this task, Forster [33] suggested that a decision-making machine should have (1) a set of ad hoc rules (or biases) to act on and (2) a set of ecologically viable environmental factors to consider.

Clark [35] extended this idea of mediating decision processes by bounded and ecological rationality through a concept he referred to as scaffolding. Clark [35] posited that human reasoning involves three aspects: (1) individual reasoning cast by some form of fast, pattern-completing style of computation (ie, bounded rationality); (2) substantial problem-solving work offloaded onto external structures and processes (eg, social and institutional structures); and (3) public language used as a means of coordinating social structures and mediating individual thought. Thus, decision making and cognition are largely dependent on the capacity to dissipate reasoning throughout the environment to reduce individual workload.

Holland and colleagues [36] added additional elements that can be useful to understand physician decision making. These elements provide a cognitive framework for problem solving, which includes two distinct schemas: pragmatic reasoning schema and problem schema. Pragmatic reasoning schemas are clusters of abstract inferential rules that characterize relations over general classes of object kinds, event relationships, and problem goals. Problem schemas are used by experts to solve routine problems, where an expert retrieves an appropriate problem schema and provides it with problem-specific parameters.

The system must also have two types of knowledge structures: mental models and condition-action rules. Holland and colleagues [36] assert that “mental models are transient, dynamic representations of particular, unique situations. They exist only implicitly, corresponding to the organized, multifaceted description of the current situation and the expectations that flow from it.” A condition-action rule can be thought of as an IF (condition)...THEN (action) statement. Together, these knowledge structures allow the mental schemas to operate in order to solve problems.

To successfully implement and use CDSSs, these mental models have to be identified. Hayek [37] stated that knowledge is not given to anyone in its entirety. This statement legitimizes why CDSSs are so important. In theory, CDSSs lessen the cognitive resources a physician needs to make decisions.

The major limitation of CDSSs is that scaffolding cannot be fully captured by computers. The environmental, clinical, and social constraints in which physicians practice are difficult to include as inputs into a CDSS. In addition, reproducing a physician’s tacit knowledge through mental models and condition-action rules is a formidable objective. Additionally, physicians must be able to support their decision and are skeptical of recommendations or claims that lack supporting evidence or transparency. The fact that CDSSs do not reveal how output decisions are made may be a driving force behind the lack of users’ acceptance.

Studies have revealed that responses to CDSSs can be unfavorable when resulting improvements in patient outcomes are inconsistent [6]. Also, some studies have reported incidents of patient harm associated with CDSS implementation [38]. Despite these findings, limited research has formally evaluated the impact of user acceptance. Based on our comprehensive review of the literature, we have found both favorable and unfavorable user acceptance to CDSSs.

If a user finds a product frustrating or perceives that the purpose of the product is to limit autonomy, the user may not use the product or do so inappropriately [39]. Vashitz et al [40] explains the consequence of loss of autonomy as reactance. Reactance is an unpleasant motivational state whereby people react to situations to retain freedom and autonomy. Reactance may exist when physicians feel threatened by clinical reminders for fear that they are losing autonomy and freedom of choice in the presence of such systems. Physicians may have the perception that these systems are meant to replace or degrade their clinical duties. Vashitz et al [40] describe how unsolicited advice may lead to a reactance state if the advice contradicts a person’s original impression of choice options.

Based on the UTAUT, user expectations need to be taken into consideration for technology to be accepted [12]. Therefore, in the design of CDSSs, the human element cannot be ignored. Reminders and alerts should be presented in such a way that the user does not find them threatening or obtrusive. User needs and expectations of a CDSS should be evaluated early and throughout the development lifecycle. For instance, Gadd [18] observed enhanced usability and usefulness by implementing usability testing in the early phases of CDSS development. They evaluated an evolving prototype of the system and observed user interactions over a 3-month period. In a series of sessions, they focused on evaluating user interactions with different sets of system features such as screen layout, input/output, and links to educational materials. Finally, they considered the user feedback on system recommendations in the design process. Compelling suggestions for system enhancements made by users during the earlier sessions influenced system development of features that were evaluated in later sessions.

Peleg et al [28] discussed the development process of their CDSS, where clinically knowledgeable users worked alongside the developers to design and implement the CDSS. They also used a lifecycle model user-centered design and evaluation process for evaluating the users’ goals/expectations, workflow, environmental constraints, and tasks. Finally, they conducted usability testing (ie, heuristic evaluation of user-interface, keystroke-level modeling, and cognitive walkthroughs) prior to implementation.

Developers of CDSSs have attempted to bottle-up the decision-making capacity of physicians and place that knowledge into a computer. Current methods to achieve this feat employ rules and machine learning algorithms. However, the lack of user acceptance has impeded CDSS use. Research has shown that consideration of users’ needs and expectations in the design of the CDSS may help overcome this obstacle. We argue that this approach is only part of the solution.

We propose that CDSSs move away from the black-box process to a more transparent method within the IPO model. Simply put, tell the physician how the computer is making the decision. If the computer can become part of scaffolded knowledge, the physician may view the computer as an aid rather than a threat or hindrance. Research supports the idea that the rules governing alerts be specified to practitioners and the information be presented based on users’ needs and expectations [41].

We propose two models to improve CDSSs development that may lead to increased utilization resulting in improved patient outcomes. First, is the user acceptance and system adaptation design (UASAD) model that aims to involve end users early in the design and throughout the development of CDSSs. Second, is replacing the current IPO model of CDSS development with the input-process-output-engage (IPOE) model that serves to “engage” the physician through CDSS process transparency.

The UASAD model demands early end-user involvement in CDSS development. User needs and expectations need to be fully realized prior to the development of a CDSS. Another consideration is to evaluate system preparedness to ensure that users can trust the security and privacy of the system. Prototypic designs should undergo an iterative design process following rigorous usability testing in a laboratory and natural setting (ie, pilot study) to ensure that the system works within the cognitive and environmental constraints with which the user functions.

Finally, user acceptance should be evaluated to ensure that the system is used appropriately. If user acceptance is not achieved above a predefined threshold, the CDSS should be reevaluated from the point of view of user needs and expectations. It should also be subjected to adaptive redesign. This process should iterate until user acceptance exceeds a predefined threshold. To illustrate this process, we have developed a UASAD model (Figure 2). The purpose of the model is to include the user as the focal point of the design process of CDSS.

The IPOE model offers users a window into the black-box IPO process. Through “engage” physicians will see how the CDSS is making decisions. The IPOE window will be called “engage” because it will present users with the rules that the machine followed to generate the output (Figure 3). Therefore, the user can make informed decisions when determining to accept or deny outputs. “Engage” will display the input, process, and output that led to the CDSS’s decision. The physician will then be able to evaluate the relevancy, validity, supporting evidence, and strength of a recommendation. Therefore, this system becomes a component of the physician’s scaffolded knowledge and enables them to act more confidently in accepting the technology and its role in their decision-making processes.

A limitation of the IPOE model is that in order for the model to work successfully, the physician has to understand the process. Processes that utilize a machine learning algorithm, such as neural networks, do not provide rules. Therefore, it is challenging to make all processes transparent.

Physicians’ tendencies to incorrectly process challenging decisions usually lead to bad clinical decisions. Most practicing physicians tend to make decisions out of their own medical experience, whereas others pursue medical consultations and filtering through the jargon of relevant research. The most effective physician, though, is the one who has the ability to utilize his clinical judgment coupled by the computerized decision support tools to leverage the power of CDSSs. Most clinicians exhibit bias when it comes to medical information that they know and, therefore, they typically focus on things that would agree with the specific clinical outcome that they want to see in their patients. Therefore, the context of using a CDSS is mandated by the efforts to decrease medical errors by utilizing existing knowledge and technology. These systems are a result of long-term scientific research to build efficient tools for physicians to supplement their clinical experience. Physicians should look at CDSSs as an added value to make the best decisions in their day-to-day practice and to better serve their patients. These systems seek to reduce medical errors by enabling the practicing physicians to make informed decisions that are both accurate and precise.

Implementation of CDSSs has demonstrated increased efficiency, reduced medical errors, and improved outcomes, but they continue to fall short of their full potential [2-9]. We believe this key shortcoming may partly be due to the lack of physician acceptance. In the past, CDSS designs have not incorporated input from physicians and do not reveal their decision-making processes. Consequently, many physicians are hesitant to accept CDSSs leading to suboptimal implementation. Here we propose two models for designing CDSSs with the goal of improving efficacy and physician acceptance. One model, UASAD, focuses on including the physician in the design process by examining user needs and expectations and usability of prototypic designs. The other model, IPOE, extends the existing IPO framework by adding an “engage” stage that displays the CDSS process to the physician. This approach allows the physician to include the CDSS as a component of their decisions while maintaining professional autonomy. There is still considerable work to be done for validating these models, yet user acceptance appears to be pertinent for successful CDSS use. Ultimately, if a physician does not accept the technology, it not only poses a threat to the use of the technology but can also pose a threat to the health and well-being of patients.