Published on 31.08.20 in Vol 8, No 8 (2020): August
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/21056, first published Jun 04, 2020.
Impact of a Commercial Artificial Intelligence–Driven Patient Self-Assessment Solution on Waiting Times at General Internal Medicine Outpatient Departments: Retrospective Study
Background: Patient waiting time at outpatient departments is directly related to patient satisfaction and quality of care, particularly in patients visiting the general internal medicine outpatient departments for the first time. Moreover, reducing wait time from arrival in the clinic to the initiation of an examination is key to reducing patients’ anxiety. The use of automated medical history–taking systems in general internal medicine outpatient departments is a promising strategy to reduce waiting times. Recently, Ubie Inc in Japan developed AI Monshin, an artificial intelligence–based, automated medical history–taking system for general internal medicine outpatient departments.
Objective: We hypothesized that replacing the use of handwritten self-administered questionnaires with the use of AI Monshin would reduce waiting times in general internal medicine outpatient departments. Therefore, we conducted this study to examine whether the use of AI Monshin reduced patient waiting times.
Methods: We retrospectively analyzed the waiting times of patients visiting the general internal medicine outpatient department at a Japanese community hospital without an appointment from April 2017 to April 2020. AI Monshin was implemented in April 2019. We compared the median waiting time before and after implementation by conducting an interrupted time-series analysis of the median waiting time per month. We also conducted supplementary analyses to explain the main results.
Results: We analyzed 21,615 visits. The median waiting time after AI Monshin implementation (74.4 minutes, IQR 57.1) was not significantly different from that before AI Monshin implementation (74.3 minutes, IQR 63.7) (P=.12). In the interrupted time-series analysis, the underlying linear time trend (–0.4 minutes per month; P=.06; 95% CI –0.9 to 0.02), level change (40.6 minutes; P=.09; 95% CI –5.8 to 87.0), and slope change (–1.1 minutes per month; P=.16; 95% CI –2.7 to 0.4) were not statistically significant. In a supplemental analysis of data from 9054 of 21,615 visits (41.9%), the median examination time after AI Monshin implementation (6.0 minutes, IQR 5.2) was slightly but significantly longer than that before AI Monshin implementation (5.7 minutes, IQR 5.0) (P=.003).
Conclusions: The implementation of an artificial intelligence–based, automated medical history–taking system did not reduce waiting time for patients visiting the general internal medicine outpatient department without an appointment, and there was a slight increase in the examination time after implementation; however, the system may have enhanced the quality of care by supporting the optimization of staff assignments.
JMIR Med Inform 2020;8(8):e21056
The waiting time at outpatient departments is directly related to patient satisfaction . Patient distrust regarding medical services increases with longer waiting time, specifically in patients visiting for the first time [ ]. Compared to those in other departments, long waiting times in the general internal medicine outpatient departments are, particularly, an issue [ ]. Low patient satisfaction may lead to poor patient safety from misunderstandings between patients and medical staff and from medical staff handling patient complaints about waiting time leaving less time for other duties such as medical care. Therefore, reducing waiting time for new patients in general internal medicine outpatient departments may play a vital role in maintaining and improving the quality of medical care. Moreover, reducing the waiting time from arrival in the clinic to the initiation of the medical examination appears to be particularly associated with a reduction of patient anxiety [ ].
Clinical documentation is time consuming, taking approximately 34% of physician working time in the outpatient department setting . Moreover, physicians can reduce their clinical evaluation time if summaries of patient histories have already been prepared prior to the examination [ ]. Such summaries can be prepared by nurses using self-administered questionnaires provided to patients in the waiting room and completed by hand; this is already widely used in hospitals across Japan. This system, however, has several limitations. First, although a handwritten self-administered questionnaire is a patient-friendly and easy method for medical personnel to collect data, it takes a long time to transfer the detailed information correctly into electronic files. Second, some patients may fill the forms only partially [ ], contributing to considerable missing data. Third, the quality of information depends on the skills of the nurse in collecting information. Finally, this system leaves nurses with less time to attend to other professional duties, including engaging in direct patient care [ ].
Automated medical history–taking devices appear to be a promising solution for reducing the time spent on transferring handwritten data into digitized form. Automated medical history taking itself has a long history since it was introduced in the late 1960s [, ]. Until recently, automated medical history taking was used outside of clinics or hospitals and took a long time to complete [ , ], but it has now been implemented in hospital and clinic waiting rooms through computing systems [ , ] and takes only 5 to 10 minutes to complete [ - ]. Its usability and acceptance by patients have been on the rise, and most patients (including older adults) can use automated medical history–taking devices without assistance [ - ]. Automated medical history taking is expected to assist physicians in developing differential diagnoses and to improve on accuracy of diagnoses, though this has not been the case previously [ - ]. Overall, computer-generated patient history recorded by an automated medical history–taking device was reported to be of higher quality, more comprehensive, better organized, and of greater relevance than patient information obtained through traditional methods of medical history taking [ ]. Moreover, it was reported to be popular among patients, enabling better communication with physicians, helping to enhance the quality of patient care and making the patients more comfortable in answering sensitive questions [ ].
However, there is a paucity of data on the efficacy of automated medical history taking in reducing waiting times. A previous study  reported that 45%-60% of physicians believed automated medical history taking could be time saving and efficient because fewer questions need be asked of the patient, less writing is necessary, the automated medical history taking provides a good basis for more detailed questioning, the history is more complete, and patients are forced to think about their problems beforehand [ ]. Although not statistically analyzed, some physicians reported average time gained using automated medical history taking was 5 minutes (ranging from none to more than 15 minutes) [ ].
Hypotheses and Study Goal
Recently, an artificial intelligence (AI)–based automated medical history–taking device, AI Monshin, was developed by Ubie Inc in Japan . AI Monshin is not only an automated medical history–taking system but also a clinical decision support system trained to suggest differential diagnoses based on AI machine learning. Based on the positive outcomes of automated medical history–taking devices [ ], we hypothesized that replacing the use of handwritten self-administered questionnaires with a new system using AI Monshin would reduce waiting time in a community hospital general internal medicine outpatient department.
We conducted a retrospective observational study using data from outpatients who visited the Department of General Internal Medicine at the Nagano Chuo Hospital. The Nagano Chuo Hospital is a medium-sized, secondary community general hospital in Nagano City, Japan and has 332 inpatient beds. The Nagano Chuo Hospital Research Ethics Committee approved the study (serial number: Nagano Chuo Byoin 20-3). The requirement to obtain written informed consent from patients was waived because of the retrospective nature of the study.
We included patients who had visited the general internal medicine outpatient department in Nagano Chuo Hospital without an appointment between 8 AM and noon on ordinary weekdays (Monday to Friday, excluding hospital holidays) from April 1, 2017 to April 16, 2020. We implemented AI Monshin in the outpatient department on April 17, 2019.
AI Monshin Tool Presentation
AI Monshin is a software that converts data entered by patients on tablet terminals into technical terms and displays it in the electronic medical record . While in the waiting room, patients enter their age, sex, and symptoms (details can be entered as free text) on a tablet. Consequently, the AI software chooses approximately 20 questions that are tailored to the patient from a pool of 3500 questions. Questions are displayed on the tablet one by one, and patients answer the questions by choosing from the items displayed. The questions are optimized according to previous answers to provide the most relevant list of potential differential diagnoses. It takes approximately 3 minutes to complete the questions [ ]. Entered data are summarized and translated into compatible medical text automatically in the patient’s electronic medical record. The top 10 possible differential diagnoses based on history generated by the AI software can be used to assist the physician during patient evaluation.
Different patient flows were applied before and after the introduction of AI Monshin. Before AI Monshin implementation, patients—upon arrival in the clinic—wrote their symptoms, past medical history, family history, social history, and medication history by hand using self-administered questionnaire forms. Upon completion of the form, patients would be interviewed by a nurse, who would check their vital signs, triage the patient, and transfer the patient's information into the electronic medical record system. A doctor would then examine the patient. During and after the examination, the doctor could also edit the patient's medical records using unstructured free text clinical notes.
After AI Monshin implementation, when patients checked in, patients were asked to enter their medical information using the tablet; 5 tablets were introduced. While 3 nurses were engaged in pre-examination interviews prior to the implementation, after the implementation a clerk staff member was hired to assist patients when using the tablets, and one of these nurses was allocated to engaging in nursing work. Clerk staff assisted those who could not use the tablet. After completing the questions on the tablet, nurses would check the vital signs of the patient and triage. Patient data were automatically summarized and translated into compatible medical text in the electronic medical record. The doctor was able to edit the text during and after clinical examination. From the patients' perspective, the difference between before and after AI Monshin implementation were experienced in the waiting and examination rooms. After the implementation of AI Monshin, the patients were only required to fill the electronic form. Patients did not need to wait to be interviewed by a nurse, which usually was the rate-limiting step in outpatient flow prior to AI Monshin implementation. Moreover, patients could see their summary on the monitor in the examination room and could use the displayed information when communicating with doctors. The patient flow after examination was the same before and after AI Monshin implementation.
Data Collection, Outcomes, and Definitions
We retrospectively collected data, including age, sex, the time of arrival in the hospital, the time of entry into the examination room, and the first registered time of the doctor’s data entry in the patient record for each individual visit. The primary outcome measure was median waiting time per patient. We collected data on waiting time both before and after AI Monshin implementation. The secondary outcome measure was the median waiting time per month. We defined the waiting time as the time between arriving in the hospital and the first recorded time of the doctor’s data entry in the patient record since the time of entry into the examination room was not recorded in all patients.
We compared the differences in median waiting time before and after AI Monshin implementation using the Wilcoxon rank-sum test. Moreover, we conducted a single-group interrupted time-series analysis [- ] to evaluate changes in median waiting time per month before and after AI Monshin implementation. We set April 2019 as the start point of implementation. In these analyses, we excluded data with the first recorded time of doctor’s data entry earlier than the time of patient’s arrival in the hospital. Statistical tests were two-tailed, and a P value<.05 was considered statistically significant. We conducted all statistical analyses using R (version 3.6.3; The R Foundation for Statistical Computing).
Population and Primary Outcome
From 21,723 eligible patient visits, we excluded 108 (0.5%) because the physicians’ recorded data entry time was earlier than the patient's arrival time (this occurred for patients who did not follow the usual reception process, such as patients who were hospital staff or patients who visited the general internal medicine outpatient department after other departments on the same day). Hence, we included data from 21,615 patients in the study—15,000 patient visits before and 6615 patient visits after the implementation of AI Monshin. Patients who visited preimplementation were significantly older than those who visited postimplementation (age: mean 58.7 versus 56.8 years; P<.001). The proportions of men and women and the distribution of arrival times were not significantly different between the two groups (). shows the distribution of waiting time in the pre (left) and postimplementation (right) groups. Both groups showed the same distribution pattern with an extremely positive skew. The median waiting time was not significantly different between the groups (74.4 minutes versus 74.3 minutes, P=.12).
|Characteristic||Preimplementation (n=15,000)||Postimplementation (n=6615)||P value|
|Age (years), mean (SD)||58.7 (19.6)||56.8 (20.2)||<.001|
|Gender, n (%)||.15|
|Men||6801 (45.3)||2930 (44.3)|
|Women||8199 (54.7)||3685 (55.7)|
|Arrival time, n (%)||.84|
|8 AM-9 AM||4369 (29.1)||1891 (28.6)|
|9 AM-10 AM||4317 (28.8)||1906 (28.8)|
|10 AM-11 AM||3489 (23.3)||1566 (23.7)|
|11 AM-noon||2825 (18.8)||1252 (18.9)|
Interrupted Time-Series Analysis
shows the trends in the number of patients and median waiting time by month from April 2017 to April 2020. The drops in waiting time in February 2020 and March 2020 (the last two dots in ) could have been partially influenced by the efforts to mitigate the risk of the spread of coronavirus disease 2019 in the waiting room. In the interrupted time-series analysis, the underlying linear time trend was –0.4 minutes per month (P=.06, 95% CI –0.9 to 0.02), the level change at April 2019 was 40.6 minutes (P=.09, 95% CI –5.8 to 87.0), and the slope change starting in April 2019 was –1.1 minutes per month (P=.16, 95% CI –2.7 to 0.4).
We added supplemental analyses for data from 9054 of 21,615 patient visits (41.9%) for whom the time of entry into the examination room was recorded in addition to the doctor's first recorded data entry. We calculated the assumed examination time as the time between patient entry into the examination room and the first recorded time of doctor’s data entry in the patient record, in 2491 of 6615 (37.7%) and 6563 of 15,000 (43.8%) patient visits before and after AI Monshin implementation (P<.001), respectively. The median assumed examination time after AI Monshin implementation (6.0 minutes, IQR 5.2) was significantly longer compared to the median assumed examination time before AI Monshin implementation (5.7 minutes, IQR 5.0; P=.003).
To the best of our knowledge, this study is the first to evaluate the associated change from an AI-based automated medical history–taking system with patient waiting times at a general internal medicine outpatient department using an extensive data set of approximately 21,500 visits. Our results showed that the median waiting times before and after AI Monshin implementation were not significantly different from one another (P=.12). Moreover, the interrupted time-series analysis also showed no significant change in median waiting time (level change: 40.6 minutes, P=.09, 95% CI –5.8 to 87.0; slope change: –1.1 minutes per month, P=.16, 95% CI –2.7 to 0.4). In addition, we observed a slight increase in the examination time (including writing the patient record), with statistical significance (P=.003), after implementing AI Monshin.
This study had several limitations. First, there was the possibility of several confounding factors (such as staff skills, demographic changes, and case complexity) and other unmeasured confounding factors affecting the results. Therefore, we conducted time-series analysis in order to better interpret the results. Second, not all patients had data for examination start time. Thus, the waiting time in this study did not represent the actual waiting time in the waiting room. Moreover, the waiting time in this study may depend on when each doctor began entering data into the patient record; some doctors may prefer to enter data during patient examination, while others may prefer to enter data after the examination.
Interpretation and Comparison With Prior Works
In our study, the use of AI Monshin did not reduce waiting time, contrary to our hypothesis for the usefulness of implementation of automated medical history taking. This negative result appears to be due to the amount of time automated medical history taking required and the characteristics of patients visiting general internal medicine outpatient departments without an appointment. As previously mentioned, automated medical history taking saves up to 15 minutes of overall patient time in the clinic when used at home in advance to the visit . This may be because doctors were able to spare enough time to grasp the complete information taken by automated medical history taking and prepare for the examination; however, in this study, automated medical history taking was used in the waiting room right before examination. In this situation, the doctors may not have been able to make use of the large amount of data taken by automated medical history taking in just a few minutes. In addition, the completeness of automated medical history taking could be paradoxically associated to more examination time in specific situations. Consequently, the implementation of automated medical history taking actually led to longer examination times. According to a previous study [ ], physicians estimated that the use of an automated medical history–taking device has the potential to become time consuming in low-complexity cases, in which the medical history is easily taken. In the setting of the small- to medium-sized hospitals in Japan, case complexity is usually low for patients visiting general internal medicine outpatient departments without an appointment [ , ]. We conducted this study in a single center (small- to medium-sized hospital) in Japan. Therefore, there may have been a selection bias since most of cases were assumed to be low-complexity cases, though no stratification of data into the degree of complexity was performed. Hence, the increase of examination time after AI Monshin implementation in this study is consistent with the assumption. This could explain why AI Monshin implementation failed to reduce patient waiting time in our study.
Although the waiting time was not reduced in this study, AI Monshin implementation may have optimized the quality of care. Previous reports  revealed that while some physicians used the same amount of time before and after the implementation of automated medical history taking, they could perform a more complete evaluation of the patient with automated medical history taking. We could not judge whether these quality changes occurred in this study because we did not survey changes such as the quantity and quality of patient-physician communication, patient satisfaction, or the accuracy of diagnosis. However, we can hypothesize that the implementation of automated medical history taking has the potential to optimize staff assignment. Indeed, after AI Monshin implementation, one out of the three nurses was replaced with a medical clerk, and thus an additional nurse was available to attend to patients. This shift in resources could have enhanced the quality of care. Moreover, because approximately half of first-visit patients revisit the outpatient department [ ], the comprehensive patient history taken by AI Monshin may enhance the quality of care for subsequent visits. Moreover, using an AI-based automated medical history–taking system may improve the quality and quantity of data records, which otherwise vary among physicians [ ], ultimately resulting in enhancement in the quality of medical care.
The implementation of an AI-based automated medical history–taking system did not reduce the waiting time for patients visiting the general internal medicine outpatient department without an appointment. In addition, we noticed a slight increase in examination time after implementation. However, the implementation may have enhanced the quality of care by supporting the optimization of staff assignments. There may have been associations between case complexity and waiting time, examination time, and description time of patients. Therefore, we envision conducting further quantitative studies that take into account case complexity and that involve medical facilities of various sizes. Testing the effectiveness of automated medical history taking in reducing consultation time and explanation time between first versus second or subsequent visits is also a target issue for future study.
There was no funding to disclose for this study. The authors thank the project team at the Nagano Chuo Hospital for their efforts in their daily work.
YH and TS designed the study. YH collected and analyzed the data. YH wrote the manuscript, and TS revised it. Both authors read and approved the final manuscript.
Conflicts of Interest
- Bleustein C, Rothschild DB, Valen A, Valatis E, Schweitzer L, Jones R. Wait times, patient satisfaction scores, and the perception of care. Am J Manag Care 2014 May;20(5):393-400. [Medline]
- Thi Thao Nguyen S, Yamamoto E, Thi Ngoc Nguyen M, Bao Le H, Kariya T, Saw YM, et al. Waiting time in the outpatient clinic at a national hospital in Vietnam. Nagoya J Med Sci 2018 May;80(2):227-239 [FREE Full text] [CrossRef] [Medline]
- Swancutt D, Joel-Edgar S, Allen M, Thomas D, Brant H, Benger J, et al. Not all waits are equal: an exploratory investigation of emergency care patient pathways. BMC Health Serv Res 2017 Jun 24;17(1):436 [FREE Full text] [CrossRef] [Medline]
- Sinsky C, Colligan L, Li L, Prgomet M, Reynolds S, Goeders L, et al. Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties. Ann Intern Med 2016 Sep 6. [CrossRef] [Medline]
- Scott D, Hallett C, Fettiplace R. Data-to-text summarisation of patient records: using computer-generated summaries to access patient histories. Patient Educ Couns 2013 Aug;92(2):153-159. [CrossRef] [Medline]
- Bachman J. Improving care with an automated patient history. Fam Pract Manag 2007;14(7):39-43 [FREE Full text] [Medline]
- Slack WV, Hicks GP, Reed CE, Van Cura LJ. A computer-based medical-history system. N Engl J Med 1966 Jan 27;274(4):194-198. [CrossRef] [Medline]
- Zakim D. Development and significance of automated history-taking software for clinical medicine, clinical research and basic medical science. J Intern Med 2016 Sep;280(3):287-299 [FREE Full text] [CrossRef] [Medline]
- Rockart JF, McLean ER, Hershberg PI, Bell GO. An automated medical history system. Experience of the Lahey Clinic Foundation with computer-processed medical histories. Arch Intern Med 1973 Sep;132(3):348-358. [CrossRef] [Medline]
- Quaak MJ, Van der Voort PJ, Westerman RF, Hasman A, van Bemmel JH. Automation of the patient history--evaluation of ergonomic aspects. Int J Biomed Comput 1987 Nov;21(3-4):287-298. [CrossRef] [Medline]
- Herrick DB, Nakhasi A, Nelson B, Rice S, Abbott PA, Saber Tehrani AS, et al. Usability characteristics of self-administered computer-assisted interviewing in the emergency department: factors affecting ease of use, efficiency, and entry error. Appl Clin Inform 2013;4(2):276-292 [FREE Full text] [CrossRef] [Medline]
- Brahmandam S, Holland WC, Mangipudi SA, Braz VA, Medlin RP, Hunold KM, et al. Willingness and Ability of Older Adults in the Emergency Department to Provide Clinical Information Using a Tablet Computer. J Am Geriatr Soc 2016 Nov;64(11):2362-2367 [FREE Full text] [CrossRef] [Medline]
- Adang RP, Vismans FJ, Ambergen AW, Talmon JL, Hasman A, Flendrig JA. Evaluation of computerised questionnaires designed for patients referred for gastrointestinal endoscopy. Int J Biomed Comput 1991 Oct;29(1):31-44. [CrossRef] [Medline]
- Benaroia M, Elinson R, Zarnke K. Patient-directed intelligent and interactive computer medical history-gathering systems: a utility and feasibility study in the emergency department. Int J Med Inform 2007 Apr;76(4):283-288. [CrossRef] [Medline]
- Smith PD, Grasmick M. Computer interviewing in a primary care office: the patients are ready. Stud Health Technol Inform 2004;107(Pt 2):1162-1165. [Medline]
- Quaak MJ, Westerman RF, Schouten JA, Hasman A, van Bemmel JH. Appraisal of computerized medical histories: comparisons between computerized and conventional records. Comput Biomed Res 1986 Dec;19(6):551-564. [CrossRef] [Medline]
- Quaak MJ, Westerman RF, van Bemmel JH. Comparisons between written and computerised patient histories. Br Med J (Clin Res Ed) 1987 Jul 18;295(6591):184-190 [FREE Full text] [CrossRef] [Medline]
- Schwitzguebel AJ, Jeckelmann C, Gavinio R, Levallois C, Benaïm C, Spechbach H. Differential Diagnosis Assessment in Ambulatory Care With an Automated Medical History-Taking Device: Pilot Randomized Controlled Trial. JMIR Med Inform 2019 Nov 04;7(4):e14044 [FREE Full text] [CrossRef] [Medline]
- Almario CV, Chey W, Kaung A, Whitman C, Fuller G, Reid M, et al. Computer-generated vs. physician-documented history of present illness (HPI): results of a blinded comparison. Am J Gastroenterol 2015 Jan;110(1):170-179 [FREE Full text] [CrossRef] [Medline]
- Arora S, Goldberg AD, Menchine M. Patient impression and satisfaction of a self-administered, automated medical history-taking device in the Emergency Department. West J Emerg Med 2014 Feb;15(1):35-40 [FREE Full text] [CrossRef] [Medline]
- Rockart JF, McLean ER, Hershberg PI, Bell GO. An automated medical history system. Experience of the Lahey Clinic Foundation with computer-processed medical histories. Arch Intern Med 1973 Sep;132(3):348-358. [CrossRef] [Medline]
- Highlighting Japan. 2019. URL: https://dwl.gov-online.go.jp/video/cao/dl/public_html/gov/book/hlj/20191201/book.pdf
- Kontopantelis E, Doran T, Springate DA, Buchan I, Reeves D. Regression based quasi-experimental approach when randomisation is not an option: interrupted time series analysis. BMJ 2015 Jun 09;350:h2750 [FREE Full text] [CrossRef] [Medline]
- Lopez BJ, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol 2016 Jun 09 [FREE Full text] [CrossRef] [Medline]
- Hoffman SJ, Poirier MJP, Rogers Van Katwyk S, Baral P, Sritharan L. Impact of the WHO Framework Convention on Tobacco Control on global cigarette consumption: quasi-experimental evaluations using interrupted time series analysis and in-sample forecast event modelling. BMJ 2019 Jun 19;365:l2287 [FREE Full text] [CrossRef] [Medline]
- Takeshima T, Kumada M, Mise J, Ishikawa Y, Yoshizawa H, Nakamura T, et al. Reasons for encounter and diagnoses of new outpatients at a small community hospital in Japan: an observational study. Int J Gen Med 2014;7:259-269 [FREE Full text] [CrossRef] [Medline]
- Kajiwara N, Hayashi K, Misago M, Murakami S, Ueoka T. First-visit patients without a referral to the Department of Internal Medicine at a medium-sized acute care hospital in Japan: an observational study. Int J Gen Med 2017;10:335-345 [FREE Full text] [CrossRef] [Medline]
|AI: artificial intelligence|
Edited by C Lovis; submitted 04.06.20; peer-reviewed by F Palmieri, I Gabashvili, J Li; comments to author 07.07.20; revised version received 28.07.20; accepted 03.08.20; published 31.08.20
©Yukinori Harada, Taro Shimizu. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 31.08.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.