Participants were selected from the Hawaii Pacific Health Medical Group for this pilot program, which was conducted in an ambulatory setting. This medical group provides care across the islands of Hawaii and includes 4 hospitals and 70 clinics located statewide. A minimum participation of 75 providers was determined per agreement between the medical group and Abridge for a 3-month trial period. All selected providers attended an instructional session and completed a prepilot survey before they were allowed to use the AI tool. Any provider not meeting these prerequisites was dropped from the program and replaced by another provider. A total of 80 providers were recruited and met the prerequisites to begin use of the AI tool in November 2024. After a familiarization and onboarding period in November, the 3-month pilot program was conducted from December 1, 2024, through February 28, 2025.

This study (2024-080) was reviewed by Hawaii Pacific Health Research Institute officials and determined to be exempt from the institutional review board’s review, as it was determined to fall under a quality improvement initiative. Participants in this study were physicians who volunteered and agreed to participate. Physicians were not required to provide written informed consent to participate. After careful ethics review and consideration by Hawaii Pacific Health, it was decided that use of this ambient AI technology during health care visits would require verbal consent from patients prior to use, and this consent was documented in the encounter note. Patient refusals were not recorded as part of this study. Per the ambient AI company, recordings and transcripts are stored on secure HIPAA (Health Insurance Portability and Accountability Act)−compliant servers for 30 days before deletion. Notes were self-audited by providers prior to submission to the medical record to ensure that there were no inaccuracies generated by the ambient AI scribe in official clinical documentation. The research team deidentified data for distribution and publication.

The conceptual framework for this study for data collection and analysis was primarily to evaluate time-saving capability, followed by burnout and workload demands. We sought to measure time savings using self-reported time in the EMR and relate those findings to actual time in the EMR using Epic Signal data. This information was then used to better understand self-reported workload demand and burnout survey results.

Data from the familiarization period were excluded from analysis as providers were added to the pilot and allowed time to learn the AI model. Safety of the AI-generated notes was determined by provider self-auditing of notes and was assessed using 5-point Likert scale responses and free-text responses from providers. To measure provider-perceived effectiveness of the AI tool, another survey with Likert scale scores was sent before and after the pilot (survey example shown in Multimedia Appendix 1). Surveys were not anonymous and were used to match provider responses from prepilot and postpilot results. Questions derived from the National Aeronautics and Space Administration–Task Load Index (NASA-TLX) scale were scored on a 21-point scale (0‐20). The rest of the survey questions were scored on a 5-point scale, including burnout and perceived ability to provide undivided attention to patients. The survey also included a self-reported measure of time spent in the EMR outside office hours per week, with response options provided in 1-hour intervals ranging from less than 1 hour to more than 10 hours. All providers who were included in the study completed all surveys, totaling 79 surveys for each of the prepilot and postpilot periods. Provider charting metrics, including “pajama time,” time in notes per appointment and per day, and number of characters used per note, were collected from Epic Signal. Epic defines pajama time as the average number of minutes per scheduled day spent on charting activities outside of 7 AM to 5:30 PM on weekdays, time outside scheduled hours on weekends, and time on unscheduled holidays. Patient satisfaction data for the proportion of patients responding “Yes, definitely” to the question, “Did this provider listen to you carefully?” on patient satisfaction surveys were collected from optional patient surveys and matched to each provider. Ambient AI scribe use metrics were collected and provided by Abridge. High use was defined as use of the ambient AI scribe on at least 60% of notes, in conformity with previously reported definitions [6]. An example of the survey for the precollection and postcollection periods is included in Multimedia Appendix 1.

For simple outcomes, descriptive statistics were used. For categorical data, including burnout rates, patient satisfaction, and provider-reported work hours outside of normal office hours, a chi-square test of association was used, and odds ratios (ORs) were calculated to test significant associations. Continuous variables, including Epic Signal metrics for time in notes per appointment, time in notes per day, pajama time, and progress note length, were compared with a paired 2-tailed Student t test. Likert scale questions were averaged for each group and compared with a paired 2-tailed Student t test, the results of which are displayed with a Gardner-Altman estimation plot to show effect size. To confirm internal consistency of the NASA-TLX data, Cronbach alpha was calculated and found to be 0.77, indicating acceptable internal consistency. A frequency distribution graph was created for all participants and fitted with a Gaussian distribution curve. Goodness of fit was determined by an R² value. A graph detailing use by specialty was created by graphing mean use with SE of the mean (SEM) for all specialties with ≥3 participants. Between-group use was then compared using a Brown-Forsythe ANOVA test with a post hoc Dunnett T3 multiple comparison test to compare each group. GraphPad Prism (version 10.6.1; GraphPad) was used for all statistical tests and graph creation.

In this pilot study conducted within a single medical group spanning multiple medical centers and clinic sites, we found that physicians who used the ambient AI scribe experienced an average reduction in time spent on notes and in the EMR. Not only did providers perceive improvements in EMR charting time, but they also reported decreased workload pertaining to clinic note-writing responsibilities. Despite these changes, there was no significant change in the distribution of symptoms of burnout by the end of the pilot period. Use among all participants was found to fit a normal Gaussian distribution, and there was no difference in use between specialties. Although high use was encouraged, 57% (45/79) of providers met high-user status.

Although we did find decreases in time spent within the EMR and on writing notes based on Epic Signal data, this time saving was relatively small, averaging only 13.6 minutes saved per day in note writing. This finding differs from the self-reported time spent in the EMR outside of clinic hours, where most providers reported hours of time saved per week after using the ambient AI scribe. This could represent an error in judgment among providers given relative differences in individual interpretation. Interpretation of this discrepancy is further limited by the inclusion of all participating providers in survey analyses, while only high users were included in the analysis of Signal metrics. This may also point to the usefulness of an ambient AI scribe as a mental offloading tool, where, despite less time saved than perceived, there is still a benefit for providers. This coincides with NASA-TLX data, where the validated index for mental workload found significant decreases in perceived workload on average [8].

This reduction in mental workload did not directly translate into reduced burnout, as the change in burnout was not statistically significant. Burnout is not reliably reduced by ambient AI scribes in studies thus far. Our study used a coarse, nonvalidated measure of burnout, which limits interpretation and is prone to bias. However, ongoing studies seek to use validated burnout metrics and to better understand the relationship between perceived mental workload reduction and burnout outcomes.

Previous literature on ambient AI scribes has been mixed in terms of outcomes and efficacy. The largest study to date, conducted by Kaiser Permanente in California, was published in 2024 and reported the results of a pilot program using Abridge among approximately 1000 providers [4]. Although this study is large, the authors performed analyses on all providers who generated at least 100 notes with the ambient AI scribe and did not stratify results by use status, thereby limiting direct comparison with our data. Despite this limitation in direct comparison, their study also found decreased time spent in the EMR and noted overall favorable remarks from physicians and patients alike.

One major limitation in the literature is the lack of data on long-term use, which these Kaiser Permanente authors recently published in a 1-year follow-up study [9]. The group found 15,700 hours of time saved in clinical documentation for AI scribe users compared to nonusers over the study year, with continued uptake and use, especially among high users, whom they define as the top third of users. Further long-term data collection will be necessary to examine the maintenance of time savings and the potential effects on loss of participants through attrition.

As this scribe is released to more providers, it can be anticipated to aid in increasing provider satisfaction, retention, and longevity, all of which have been declining critically short nationwide [10]. How ambient AI scribes affect burnout is also contested, with some studies finding significant improvements while others, including this study, have found no significant difference [11-14]. Although benefits for burnout remain contested, perceived reductions in workload are consistent across studies and are in agreement with our findings [11,12]. Additionally, subgroup analyses of outcomes for different specialties remain limited, especially outside the realm of primary care [15].

A secondary outcome of this study was to determine the safety of this ambient AI scribe to ensure that patients and physicians are protected. Providers considered the ambient AI scribe safe to use for their patients, as long as there was examination of each note prior to submission, because, although infrequent, hallucinations and omissions occurred. Ensuring that notes encompass the critical visit details without omissions or hallucinations is a critical metric for all stakeholders. Our study used provider self-auditing and self-reporting on random notes to examine the safety profile. Independent validation of ambient AI safety is difficult because the underlying development of LLM algorithms is proprietary and varies between products, so we must rely on output metrics. Other studies have attempted to define the safety of these products but largely reach similar conclusions: even when these tools are effective, there are noticeable rates of errors, with the most common being omissions [16-18].

Patient satisfaction is also imperative to assess during this rollout to ensure that the introduction of new technology does not hinder or negatively affect the patient-physician interaction. Our results showed no change in patient satisfaction scores for providers participating in the pilot. Although we found that high users had a nonsignificant increase in the “provider listened to me” score on patient surveys, more data collection over time with longer implementation will be more useful for the interpretation of this effect. Other literature on patient perception of the impact of AI scribes on clinic visits is limited. One study surveying 21 patients at a single clinic found that 71% of patients said they felt they spent more time with their physician, while only one patient said they spent less time [4].

There are several limitations to this study. First, we rely, for several metrics, on self-reported survey responses, which are prone to bias. Although we attempted to substantiate some self-reported responses with objective measures, such as time spent in notes outside of work, this could not be accomplished for all metrics. This is particularly true of burnout. Second, self-auditing notes for safety are similarly prone to bias. This form of note evaluation is suboptimal for an in-depth analysis of note-writing capacity for this ambient AI. However, we did not include more thorough objective measures of note-writing analysis as this was not the primary objective of the study, and this area has already been studied for this technology platform [18,19]. Third, despite encouragement to participate, only 45 of the 79 participants met high-user status by using the ambient AI scribe in at least 60% of notes. This decreases the power of the study and points to barriers to uptake that are multifactorial and should be subject to further study.

The findings presented here offer opportunities for further study on ambient AI in the clinical workspace. One opportunity is to expand the cohort to increase the power of the study. By increasing the number of participants, we will also increase the representation of different specialties for subgroup analyses to better understand whether this technology is better suited to specific roles. Expanded power and use of a validated instrument for assessment of burnout will aid in the analysis of the interplay between workload reduction and burnout outcomes [20]. Furthermore, recording and analyzing patient refusals of ambient AI scribe use during encounters, as well as the reasons for those refusals, may elucidate a better understanding of how incorporation and patient education can be improved. Next, further analysis of outcomes by use group (high user vs low user) would improve understanding of the application of this technology and its association with work-life benefit outcomes. Additionally, our providers noted limitations with non-English speaking encounters where there may be an increased occurrence of errors. Noted problems with non-English encounters include the difficulty of the ambient AI scribe picking up contextual exchanges, translating details, and reducing the need for extensive editing in non-English languages. Further study of limitations in these scenarios will aid in improved product development and application to patient care. Finally, ambient AI scribes represent a rapidly evolving landscape with continuous changes and improvements across platforms and interfaces. These changes are anticipated to enhance usability while improving workflow. For example, queuing orders captured during discussion and recommending appropriate evaluation and management service levels are planned integrations that could improve time-saving capability, reduce mental workload, and mitigate burnout, especially since poor EMR usability has been tied to burnout [10].

This pilot study of 79 providers in a single medical group evaluated the implementation of the Abridge ambient AI scribe and found a decrease in note-writing time in the clinic. There was no significant difference in reported burnout symptoms, although self-reported workload did decrease.