To address the challenge of documentation burden in the ICU setting, we aimed to explore the use of AI-assisted ambient scribes. This project was divided into 2 phases. Phase 1 focused on customizing the prompt for the LLM component of the AI-assisted scribes for ICU rounds, while phase 2 involved evaluating the quality of the generated notes using a customized prompt. Figure 1 illustrates the process map of our project.

We used software-generated and manually reviewed transcripts of audio recordings from simulated rounds designed by domain experts. The underlying cases were derived from real-world examples. The simulations took place in a real ICU environment to ensure high fidelity. A detailed description of this process can be found in an earlier paper by Bordley et al [20] from our lab.

Using the percentages and themes of errors from phase 1 (described above), the prompt underwent 3 rounds of updates: V1→V4. Phase 2 evaluations involved comparing the accuracy and percentage of errors using prompt V4 and models (M1 and M6) across 5 separate cases (C1-C5), as described earlier. A Shapiro-Wilk test was used to check the normality of the data. Boxplots were created to check for outliers. A paired 2-tailed t test (data were normally distributed, and there were no outliers) was performed to compare the accuracy and errors (including subtypes) between M1 and M6. Bonferroni correction was applied where applicable for multiple comparisons. To compare the overall (combined M1 and M6) percentage of error subtypes, a Friedman test followed by Bonferroni correction with multiple comparisons was used, as the data were paired. Mean error severity scores were compared between M1 and M6 using a paired 2-tailed t test (normally distributed, no outliers). The chi-square test was used to compare the error severity categories between models M1 and M6. Spearman correlation analysis was conducted between accuracy and note length. A Wilcoxon signed-rank test was conducted to compare the average of PDQI-9 scores between M1 and M6 outputs. A P value of <.05 was considered statistically significant. All descriptive statistics are quoted as mean/percentages (SEM). All analyses were performed using Python (version 3.12.1). Figures were generated with the Python packages Matplotlib and Seaborn.

The study was deemed exempt by the institutional review board (7551) because only synthetic data were used to develop ICU rounds simulations. The synthetic data were derived from real-world cases covered under Institutional Review Board 9943.

Five separate models were evaluated in this phase of the study with different versions of the prompts (Table 2 and Figure 2). At each stage, team members used a combination of quantitative and qualitative assessments of the errors (Figures 2 and 3 and Table 1) to decide which models would proceed. Most errors were related to omissions, and qualitative assessment revealed which sections and subsections of notes were most affected. These insights guided modifications to the prompts to address those errors. Models M2 and M3 performed poorly and were dropped early. M4 initially performed well but showed inconsistent results and a decline in performance after updates. Both M1 and M5 continued to perform well and responded positively to prompt updates at each stage. The team decided to proceed with M1 for the next phase of testing, as M5 was discontinued later in the study.

Based on phase 1 results, all phase 2 evaluations proceeded with version 4 of the prompt. Model M1 performed consistently across new cases, with an accuracy of 69.12% (95% CI 69.3%‐77.9%; Table 3 and Figure 4). In comparison, model M6, a direct upgrade of model 1 (Multimedia Appendix 1 for model description), performed better across all 5 cases, achieving an overall accuracy of 80.3% (95% CI 75.2%‐85.5%; P=.04).

Regarding error subtypes, combined (both M1 and M6) errors of omission were the most common (Figure 5) with an average of 15.5 (SEM 2.7%, 95% CI 9.4%‐21.6%), followed by partial errors at 7.2 (SEM 0.92%, 95% CI 5.1%‐9.3%), with errors of commission being the least common at 2.6 (SEM 0.7%, 95% CI 1.1%‐4.1%). Overall comparison of combined errors was statistically significant (P=.001), with subcomparison statistically significant only between omission and commission (adjusted P=.02). In terms of absolute counts, the errors of omission occurred at an average of 16 (SEM 2.85) errors per case, while partial errors occurred at 7.3 (SEM 0.89) errors per case, and errors of commission at 2.8 (SEM 0.76) errors per case. Comparison of error subtypes between the 2 models showed that only omission errors (21.60% vs 9.32%; adjusted P value =.03) were statistically significantly different between M1 and M6. On a deeper analysis of the errors, we compared which parts of the SOAP note structures contained the most errors. For M1, we observed that 5.08%, 2.75%, 1.95%, and 21.09% of the errors belonged to the Subjective, Objective, Assessment, and Plan categories, respectively. Similarly, for M6, these numbers were noted: 5.06%, 1.24%, 2.63%, and 10.76%, respectively. Further qualitative analysis of errors, with notable examples (Table 4), showed that they were related to historical data, vitals, lab data, imaging results, problems, differentials, and plans. M1 and M6 had similar errors of omission and partial errors, with the primary difference being quantitative: M6 had a lower proportion, as noted earlier. In contrast, errors of commission varied significantly by models, as noted in the examples in Table 4. Furthermore, most partial errors were omission-related.

The analysis of error severity showed that most errors (n) fell into the no harm or low harm categories (Figure 6). For the moderate-to-fatal harm category, M1 had errors at 0.2 (SEM 0.2) per case, whereas M6 had 0 (SEM 0). Furthermore, within the moderate-to-fatal category, no errors were classified as potentially severe or fatal. M1 had no harm errors at a rate of 6.6 (SEM 1.1) per case, compared to 5.0 (SEM 0.89) per case for M6, while low harm errors occurred at 25.2 (SEM 3.54) per case for M1 and 15.2 (SEM 1.85) per case for M6. The differences in severity between M1 and M6 were not statistically significant (P=.55). Table S2 in Multimedia Appendix 1 provides examples of each harm score category and their descriptions. Finally, the difference in average error severity scores between M1 and M6 (mean 0.61, SEM 0.03 vs mean 0.53, SEM 0.05; P=.10) was not statistically significant either.

In terms of note structure, the average note length for M1 and M6 outputs was 534.6 (SEM 35.7) and 1093.8 (SEM 57.5) words, respectively. Overall, a strong correlation was found between note length (r=+0.83; P=.003) and accuracy. Furthermore, note length had a very strong statistically significant negative correlation with errors of omission (r=−0.83; P=.003), and a strong positive correlation, albeit statistically not significant, with errors of commission (r=+0.62; P=.06). In contrast, there was no correlation with partial errors (r=−0.15; P=.66). Of note, when evaluated within models, the only significant correlation was between errors of commission and note length, within model M6 (r=+0.90; P=.04). Finally, in terms of overall readability and usefulness, M6 was better than M1 based on PDQI-9 scores, but the difference was not statistically significant (M1=35.8, SEM 0.78 vs M6=38.3, SEM 0.53; P=.06).

This study is the first of its kind to examine the feasibility of using AI-assisted ambient scribe technology in the ICU setting. Our findings show that the LLM component of the AI-assisted ambient scribes can be customized to capture interprofessional ICU rounds and produce a singular document aligned with ICU workflows. While the current literature is limited on the use of AI scribes in the ICU, our results suggest that the use of customized AI-assisted scribes in the ICU setting can generate similar outputs compared to outpatient AI-assisted scribe apps [26-28]. Our results demonstrate that prompt engineering plays a crucial role in improving note quality and customizing them for specific settings and formats. However, the variations observed between different LLMs using the same prompt indicate that technological optimization must address both areas. This aligns with other studies showing high variability in LLM models’ performance with identical prompts [29-31]. Our study highlights the need to further investigate the use of AI-assisted scribes in ICU settings and the potential benefits of conducting additional research to validate our findings in a larger, more diverse cohort and to evaluate the accuracy of automated speech recognition components in scribes and their correlation with note errors.

Existing documentation practices are also prone to errors. Data omission is particularly common, with 100% of ICU patient presentations containing at least one omission in 1 study [32]. In at least 1 study, when physician trainees prepared prerounding notes and verbal presentations, 22.9% of the data were missing from written artifacts and 42.4% from the oral presentations [32]. Another study shows laboratory data communication during ICU rounds was similarly problematic, with 96% of patient presentations including at least 1 laboratory misrepresentation and 38.9% of all audited laboratory data being inaccurately communicated [33]. Most of these errors were omissions rather than incorrect values, and only 7.8% were detected by the rounding team [33]. Our results show the viability of AI-scribe technology for accurately reflecting what is being discussed. However, as noted in these prior studies, interprofessional rounds in ICU settings are far from perfect, underscoring the need for standardization. In our study, we observed that most errors occurred in the Plan section, likely reflecting greater ambiguity in assessment and planning. However, the errors were in the objective or lab data in these prior studies. This further emphasizes the need to standardize verbal presentations; otherwise, we would just be compounding errors. Conversely, AI scribes should create notes that are free of copy-and-paste information, thereby reducing errors related to these mechanisms in the electronic health records [34]. In studies outside ICU settings, prior research has reported error rates ranging from 7% to 21% in clinical documentation, whether derived from dictation software (office notes, discharge summaries, and procedure notes) or patient-reported data (ambulatory notes) [35,36].

Upon closer examination of errors in AI-assisted scribe documentation output, we found that most originated from omissions. Therefore, our main focus in prompt engineering was to reduce missing elements in the generated note. One method to accomplish this was to add a gold-standard template to the prompt. The Plan section was divided into organ-based subsections, which helped identify issues related to specific areas, such as psychiatric problems including delirium and sedation, that were not always captured under the neuro subsection in earlier versions of the prompt. However, by explicitly including subsections for most organ systems in the note template, we observed that models sometimes generated plans for endocrine or hematology-oncology sections even when these were not discussed in the transcript. Although the generated content was not always incorrect or clinically significant, as shown in our qualitative results, for example, “anemia noted” when hemoglobin numbers are in the anemia range, this approach appeared to carry a risk of errors of commission, such as hallucinations or fabrications. Some errors could have a greater clinical impact, including statements such as “cause of anemia” or discussions of “transfusions.” This indicates a trade-off when using a more detailed template in the prompt to better capture data elements. Of course, this trade-off could be eliminated by using strategies to optimize the prompt to reduce errors of commission. This remains an area to be explored in future work.

To enhance completeness and reduce errors of omission, additional prompt engineering procedures included adding “New Orders,” “Discontinued Orders,” and a “Medications” subsection at the end of the note. This resulted in significantly improved performance. Interestingly, the transcripts also included a discussion at the end where the team listed the new and discontinued orders and medications for the day. This presents an opportunity for clinicians to adjust workflows to bring rounds closer to the note templates, thereby further improving transcript and note accuracy. However, future studies should examine the use of the ambient scribe in real-world ICU settings to ensure it does not impose additional tasks or cognitive burdens on the teams. This is especially important, given that the current safeguard against errors in AI scribe notes is manual review by clinicians, which adds an additional burden on clinicians and could lead to automation complacency and/or bias [37-42].

Besides prompt engineering, we found that upgrading LLMs also greatly enhanced performance. Once the prompt was set, using M6 (the larger model) yielded significantly better performance than the earlier version, M1. Part of this difference could be explained by M6’s ability to generate longer notes. We observed a strong overall correlation between note length and accuracy (and errors of omission); however, this correlation is primarily driven by between-model differences rather than a continuous within-model trend. Interestingly, there was a strong positive correlation between note length and overall errors of commission (primarily within model M6), although their impact on accuracy (and total error rate) was not large, as errors of commission constituted a much lower percentage of total errors. This shows a small trade-off between errors of omission and errors of commission depending on the model’s thoroughness or the length of the note. Although longer notes (model-dependent) appear to yield lower error rates, this could affect workflows, further burdening clinicians who would have to manually review longer notes. Ironically, as noted in another study [34], copy-and-pasted information increases the length of the notes, and AI-generated notes, in theory, should not contain copied-and-pasted information. Thus, generating longer notes by AI to reduce omissions might not have as much impact on workflows as anticipated and presents another opportunity for future research. This is evident in the perceived usefulness of these outputs in PDQI-9 scores, where the differences between model outputs are not statistically significant; in fact, the trend favors M6.

Another consideration when using ambient scribes in the ICU is privacy and confidentiality [41,42]. When assessing the feasibility of ambient listening devices in the ICU using the clinical rounds’ structure, which is generally an open environment (as opposed to office-based visits), we need to be mindful that these devices will capture speech from multiple clinicians, families, and ancillary staff. This has implications for consent procedures [41]. For example, would everyone in the ICU have to consent to AI scribes? What about people entering the ICU after rounds have started? Each state’s wiretapping laws may further complicate these regulatory issues [43]. Other regulatory considerations for the use of AI scribes include HIPAA compliance and data security [42], such as where these conversations are stored and for how long. Implementing institutional data governance policies, recording practice protocols, and keeping up with evolving guidance for AI-enabled tools is essential [44].

While the absolute number of individual errors was significant for both models, the severity of these errors was mostly low, categorized as no harm to low harm potential. Additionally, we observed that the error severity scores showed little difference between the 2 models. These results are encouraging, though it is too early to draw definitive conclusions given the small sample size and limited variation in clinical problems in our dataset.

Interestingly, in a few instances, LLMs seemed to extract the correct information from contextual clues, even when errors occurred in the transcripts. This is likely a probabilistic inference based on LLM training data. This could be a strength in ICU or inpatient settings, where open environments and ambient noise can affect transcript quality. However, the autonomous correction of clinical data without flagging it for a human being should be recognized as a potential safety risk. Incorporating safeguards, such as uncertainty signaling or explicit error flagging, could help reduce the cognitive burden of revisions needed after AI scribes generate a note. Since our data lacked sufficient errors in the transcripts to thoroughly analyze model behavior, this remains an area for future research.

There are some limitations to our study. First, we did not investigate whether automated speech recognition software impacts transcript quality in ICU settings. While this is important, it was out of scope for this initial pilot to assess the feasibility of customizing the prompt for the ICU-specific setting. We plan to address this in our future work on this project. Second, the small sample size and the use of synthetic cases lacking significant clinical variation might overestimate the accuracy and would limit the generalizability of our findings. We also kept case C1 (used for prompt tuning in the first phase) as a baseline in the second phase for comparison; this could have somewhat inflated the quantitative results, as the prompt was optimized specifically for case 1. Since our synthetic cases were developed from real-world ICU rounds, they partially captured the nonlinear reasoning and diagnostic uncertainty expected in the ICU setting. Additionally, audio simulations were developed using multiple speakers and a simultaneous multiple-team rounding structure, capturing acoustic noise. However, as stated earlier, the goal of this study was to explore the feasibility of our approach in ICU settings. This is early work, and a continuation of our project will build on this. As the inferential statistics are underpowered, our findings should be interpreted as strictly preliminary and hypothesis-generating rather than definitive evidence of performance across ICU populations. Third, we did not perform a head-to-head comparison with standard error rates in ICU notes. The 80% (approximately) accuracy should not be interpreted as evidence of superiority or inferiority relative to current clinical documentation practices. It is a reflection of agreement with the expert-defined content in a controlled environment. The goal of this paper was to assess the minimum safety level before we test this technology in a real-world environment. Fourth, we had only moderate agreement (κ=0.58), reflecting subjectivity in defining errors within the ICU notes. We addressed this by manually reviewing each disagreement together and revising the classification based on the combined consensus, thereby reducing the impact on measurements. This also reflects the inherent complexity of ICU documentation rather than just subjectivity among reviewers. Clinicians often differ in their judgments of the notes’ relevance and completeness. These reliability metrics should be interpreted as approximate indicators rather than definitive measures, suggesting the need to improve them in the future as well. Fifth, the evaluators were not blinded, which could have added confirmation bias, especially in subjective metrics such as partial errors (despite reviewing disagreements together) and PDQI-9 scores. Sixth, we used only models (Llama and Claude) available on the HIPAA-compliant Amazon Web Services Control Tower at Oregon Health & Science University. Since we empirically used Claude in phase 2, our results should not be generalized to all other LLMs. We noticed a trend toward larger language models performing better (M6 vs M1); however, it remains unclear whether this holds for other model classes. Future work is needed to evaluate prompt transferability to other LLMs. Seventh, our study did not evaluate whether M6’s superior performance was due to inherent improvements in reasoning or simply because its outputs were twice as long as those of M1. There is an apparent trend toward a trade-off between the conciseness and completeness of these AI-generated clinical notes. Future evaluations incorporating normalization strategies, such as accuracy per unit length of the note or trials of different prompts urging smaller models to write longer notes, could be helpful.

Our future goals build upon this work in three key areas: (1) evaluating AI scribes in diverse ICU cases and clinical conditions; (2) assessing the accuracy and quality of speech recognition, transcripts, and notes; and (3) studying the real-world use of ambient AI scribes to understand their impact on workflows and any potential unintended consequences.

In conclusion, we examined the use of AI scribes in the ICU setting. By combining prompt engineering with multiple LLMs, we successfully generated ICU-specific notes with fairly high accuracy. The overall quality and usefulness received high ratings from team members with domain expertise. Analyzing the different types of errors offered valuable insights into areas where efforts could be directed to enhance the quality of these notes. Since AI scribes have seldom been used in ICU environments, our study offers an important starting point and valuable insights for further exploration of this application.