Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 15.05.20 in Vol 8, No 5 (2020): May

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/17647, first published Dec 31, 2019.

This paper is in the following e-collection/theme issue:

    Original Paper

    Clinical Desire for an Artificial Intelligence–Based Surgical Assistant System: Electronic Survey–Based Study

    1Department of Obstetrics and Gynecology, Seoul National University College of Medicine, Seoul, Republic of Korea

    2Department of Surgery, Seoul National University College of Medicine, Seoul, Republic of Korea

    3Department of Urology, Seoul National University College of Medicine, Seoul, Republic of Korea

    Corresponding Author:

    Hee Seung Kim, MD, PhD

    Department of Obstetrics and Gynecology

    Seoul National University College of Medicine

    101 Daehak-Ro

    Jongno-Gu

    Seoul, 03080

    Republic of Korea

    Phone: 82 02 2072 4863

    Email: bboddi0311@gmail.com


    ABSTRACT

    Background: Techniques utilizing artificial intelligence (AI) are rapidly growing in medical research and development, especially in the operating room. However, the application of AI in the operating room has been limited to small tasks or software, such as clinical decision systems. It still largely depends on human resources and technology involving the surgeons’ hands. Therefore, we conceptualized AI-based solo surgery (AISS) defined as laparoscopic surgery conducted by only one surgeon with support from an AI-based surgical assistant system, and we performed an electronic survey on the clinical desire for such a system.

    Objective: This study aimed to evaluate the experiences of surgeons who have performed laparoscopic surgery, the limitations of conventional laparoscopic surgical systems, and the desire for an AI-based surgical assistant system for AISS.

    Methods: We performed an online survey for gynecologists, urologists, and general surgeons from June to August 2017. The questionnaire consisted of six items about experience, two about limitations, and five about the clinical desire for an AI-based surgical assistant system for AISS.

    Results: A total of 508 surgeons who have performed laparoscopic surgery responded to the survey. Most of the surgeons needed two or more assistants during laparoscopic surgery, and the rate was higher among gynecologists (251/278, 90.3%) than among general surgeons (123/173, 71.1%) and urologists (35/57, 61.4%). The majority of responders answered that the skillfulness of surgical assistants was “very important” or “important.” The most uncomfortable aspect of laparoscopic surgery was unskilled movement of the camera (431/508, 84.8%) and instruments (303/508, 59.6%). About 40% (199/508, 39.1%) of responders answered that the AI-based surgical assistant system could substitute 41%-60% of the current workforce, and 83.3% (423/508) showed willingness to buy the system. Furthermore, the most reasonable price was US $30,000-50,000.

    Conclusions: Surgeons who perform laparoscopic surgery may feel discomfort with the conventional laparoscopic surgical system in terms of assistant skillfulness, and they may think that the skillfulness of surgical assistants is essential. They desire to alleviate present inconveniences with the conventional laparoscopic surgical system and to perform a safe and comfortable operation by using an AI-based surgical assistant system for AISS.

    JMIR Med Inform 2020;8(5):e17647

    doi:10.2196/17647

    KEYWORDS



    Introduction

    Artificial intelligence (AI) has been rapidly developing in recent years, and relevant research is being actively conducted in the health care field through deep learning and big data technology [1]. AI applied in the medical area can be divided into the following two categories: virtual and physical AI. Virtual AI includes the programs that can help clinical diagnosis, whereas physical AI involves smart operating rooms, nanorobots, and patient-assistance systems [2]. In particular, physical AI in the operating room can assist the operator or replace the assistant during surgery [2,3]. For instance, the da Vinci surgical system, which is the first computer-based robotic surgical system approved by the US Food and Drug Administration in 2000, has been widely used for minimally invasive surgery, including laparoscopic surgery. The demand for the robotic surgical system is rapidly increasing in the surgical areas of gynecology, general surgery, and urology [4]. This increase in demand is due to reduced surgeon fatigue and improved surgical access through ergonomic instruments and three-dimensional imaging [4,5].

    However, the current robotic surgical system still depends on coordination of the human eye and hand, which is insufficient in terms of autonomy or interaction [6,7]. In particular, the injection of carbon dioxide and insertion of trocars into the peritoneal cavity are still performed by surgeons without the aid of a robotic surgical system, and the laparoscopic camera and instruments are adjusted manually to the target by surgeons. Thus, an automated robotic surgical system that is better than the current master-slave approach may be expected to reduce human error and thereby improve the quality of surgery. Up to now, relevant studies have mainly focused on the development of robots capable of performing short surgical tasks, such as knot tie and needle insertion [8,9], and the application of voice interaction technology during surgery may be one of the crucial elements that should be developed in an AI-based surgical assistant system [10-12].

    Nevertheless, high medical cost may be one of the barriers to the adoption of an AI-based surgical assistant system [13], and it is not yet know how this system will improve the quality of surgery or reduce human resources effectively. Therefore, we conceptualized AI-based solo surgery (AISS) that was defined as laparoscopic surgery conducted by only one surgeon with support from an AI-based surgical assistant system and considered the clinical desire for AISS via an electronic survey (e-survey).

    An e-survey has been a common method of research in human and social sciences since the 1990s. In the case of research using a web-based questionnaire, it is possible to attach pictures or materials in order to avoid response omission as much as possible and avoid inconsistent or out-of-frame results. Besides, data can be effectively organized and archived without paper resources, and distribution via email can be quickly done through a URL [14]. Moreover, by distributing the web questionnaire via email, it is possible to limit the target respondents to people belonging to a specific community so that the questionnaire survey is conducted for experts in the relevant field.

    Therefore, we performed an e-survey to investigate the clinical desire for an AI-based surgical assistant system for AISS as compared with the current laparoscopic surgical system and to determine the reasonable cost of such an AI-based surgical assistant system for AISS.


    Methods

    Survey

    We surveyed gynecologists from the Korean Society of Obstetrics and Gynecology, urologists from the Korean Urologic Association, and general surgeons from the Korean Surgical Society between June and August 2017 through nownsurvey (ELIMNET Co, Ltd) [15], a commercially available e-survey platform. In this survey, the AI-based surgical assistant system for AISS was considered to have the following functions: camera automatic recognition and operation function through voice commands; action as an assistant by manipulating surgical instruments through automatic screen recognition and voice commands; and smart storage for recognizing, indexing, and storing surgical procedures while recording specific events. There were a total of 13 questions that included six items about the responder’s experience, two about limitations of the conventional laparoscopic surgical system, and five about the clinical desire for an AI-based surgical assistant system for AISS (Table 1). We estimated that 5000 gynecologists, 7000 general surgeons, and 2500 urologists would participate in the survey. This study was approved by the Institutional Review Board of Seoul National University Hospital (approval no: 1910-131-1072).

    Table 1. Questionnaire details.
    View this table

    Data Analysis

    We analyzed each question by using descriptive statistics. Additionally, we analyzed all the respondents, and the response rate was 3.5%. Each item in the questionnaire was stratified according to the surgeons’ fields as follows: gynecologists, urologists, and general surgeons. Categorical variables were analyzed with the chi-square test or Fisher exact test using the statistical software SPSS 20.0 (IBM Corp, Armonk, New York, USA). A P value <.05 was considered statistically significant.


    Results

    Experience

    Table 2 shows the demographic data of the responders. A total of 508 people responded to the questionnaire, and there were 278 gynecologists, 173 general surgeons, and 57 urologists. Among the three surgeon fields, most of the urologists (49/57, 86.0%) worked at a university hospital, whereas relatively many gynecologists (67/278, 24.1%) worked as general practitioners. Moreover, most of the urologists (43/57, 75.4%) performed laparoscopic surgery in less than 10 cases per month, whereas relatively many general surgeons performed laparoscopic surgery in 31 or more cases per month (40/173, 23.1%). In terms of the number of assistants during laparoscopic surgery, 38.6% (22/57) of urologists required one or less assistant, whereas 90.3% (251/278) of gynecologists required two or more assistants.

    In terms of the preferred assistant during laparoscopic surgery, most of the urologists (33/57, 57.9%) preferred fellows, whereas many general surgeons (76/173, 43.9%) preferred physician assistants (Figure 1). With regard to the importance of the skillfulness of assistants, who manipulate cameras or instruments, for successful laparoscopic surgery, most of the responders indicated “very important” or “important,” regardless of the surgeon field. Although the trend was similar among the three surgeon fields with regard to the camera assistant, general surgeons (33/173, 19.1%) relatively underestimated the importance of the skillfulness of instrument assistants as compared with gynecologists (93/278, 33.5%) or urologists (18/57, 31.6%) (Figure 2).

    Table 2. Demographic data.
    View this table
    Figure 1. Comparison of assistants preferred during laparoscopic surgery.
    View this figure
    Figure 2. Comparison of the importance of the skillfulness of (A) camera and (B) instrument assistants.
    View this figure

    Limitation

    Table 3 shows the responses to questions on the surgeons’ discomforts related to inexperienced camera and instrument assistants for the conventional laparoscopic surgical system. With regard to the camera assistant, 84.8% (431/508) of the responders were unsatisfied with unskilled movement of the camera in the intended direction. In particular, gynecologists (69/278, 24.8%) had more complaints about contamination of the camera lens by blood or body fluid as compared with general surgeons (26/173, 15.0%) or urologists (6/57, 10.5%). With regard to the instrument assistant, 59.6% (303/508) of the responders were unsatisfied with unskilled movement of the instruments in the intended direction. In particular, general surgeons (103/173, 59.5%) had more complaints about tissue damage or bleeding by inappropriate traction and urologists (24/57, 42.1%) had more complaints about collision between the instruments as compared with the other surgeons. On the other hand, gynecologists (32/278, 11.5%) had more complaints about swaying of the instruments as compared with the other surgeons.

    Table 3. Surgeons’ discomforts regarding the conventional laparoscopic surgical system.
    View this table

    Desire

    Table 4 depicts the functions that should be included in an AI-based surgical assistant system for AISS to overcome the limitations of the current laparoscopic surgical system. More than half of the responders preferred intuitive and easy maneuverability (308/508, 60.6%), a demister and self-cleaning system for the laparoscopic camera lens (326/508, 64.2%), and safety for minimizing tissue damage (279/508, 54.9%). In particular, more urologists (29/57, 50.9%) desired fast running by minimizing time delay as compared with gynecologists (86/278, 30.9%) and general surgeons (67/173, 38.7%). However, interest in the autosave or voice command system for special events during the operation was the lowest among the three surgeon fields. In terms of the possibility that the AI-based surgical assistant system for AISS can replace the functions of assistants, about 40% (199/508, 39.1%) of responders expected it to substitute 41%-60% of the existing workforce (Figure 3).

    When asked about the purchase intention and reasonable price to buy the AI-based surgical assistant system for AISS, 83.3% (423/508) of all responders wanted to buy the system. The most common reason for wanting to buy the system was the comfort of laparoscopic surgery (257/508, 50.6%). In particular, general surgeons had a relatively strong desire to decrease the burden of repetitive training for assistants, whereas they had less interest in the reduction of the operation time by purchasing the AI-based surgical assistant system for AISS as compared with gynecologists. Regarding the reasonable price for the system, 29.7% (151/508) of the responders had a willingness to pay US $30,000-50,000 (Table 5).

    Table 4. Functions that should be included in an artificial intelligence–based surgical assistant system.
    View this table
    Figure 3. Expectations about how much an artificial intelligence–based surgical assistant system can replace the existing workforce.
    View this figure
    Table 5. Purchase intention and reasonable price to buy the artificial intelligence–based surgical assistant system.
    View this table

    Discussion

    Principal Findings

    This study involved a survey about the clinical desire for an AI-based surgical assistant system for AISS among surgeons who currently perform laparoscopic surgery. In this survey, we identified the importance of assistants and the discomforts with the conventional laparoscopic surgical system and determined surgeons’ expectations and demands for new AI-based robotic surgery aids.

    Experience

    In terms of experience, gynecologists were more likely to have two assistants than general surgeons and urologists. The reason is that gynecologists may use a uterine manipulator frequently during laparoscopic gynecologic surgery [16]. Therefore, gynecologists commonly require two or more assistants for laparoscopic surgery, including two assistants who hold a laparoscopic camera and a uterine manipulator.

    On the other hand, urologists’ preference for fellows as surgical assistants could be related to more common practice in university hospitals. Moreover, urologists can be less dependent on residents during surgery, which may be similar for general surgeons who prefer physician assistants as surgical assistants. Furthermore, most of the responders valued the skillfulness of surgical assistants who manipulate the laparoscopic camera and instrument assistants, because the extent of assistant experience may be closely related to the operation time and complication rate [17]. Recently, in the Republic of Korea, owing to the implementation of the special act regarding an 80-hour workweek for residents, their working time has reduced, and thereby, the number of cases of surgical training has reduced [18]. In contrast, physician assistants are still useful for coordination in the operating room because of their high level of proficiency based on repetitive work [19]. Therefore, most surgeons seem to prefer fellows or physician assistants who are proficient in laparoscopic surgery rather than residents or interns because of their skillfulness as surgical assistants in the Republic of Korea.

    Limitation

    In terms of limitation, most of the surgeons felt uncomfortable with camera assistants when they showed unskilled movement of the camera in the intended direction and instrument assistants when they showed unskilled movement of the instruments in the intended direction. This result is consistent with the finding that most of the surgeons considered the skillfulness of surgical assistants as “very important” or “important,” regardless of the field.

    Desire

    In terms of desire, the essential functions desired to be present in an AI-based surgical assistant system for AISS were intuitive and easy maneuverability, a demister and self-cleaning system for the laparoscopic camera lens, and safety for minimizing tissue damage. Interestingly, only 10%-20% of surgeons complained about discomfort regarding the camera lens or foreign objects, whereas a high percentage of surgeons desired a self-cleaning system for AISS. These findings seem to be associated with the role of surgical assistants in camera cleaning when using the conventional laparoscopic surgical system, which is perceived as an essential function by the operator, and the absence of an uncomfortable feeling with the current system.

    Notably, more than 80% of the responders intended to buy the AI-based surgical assistant system for AISS, and the reasons for buying it were comfort of laparoscopic surgery and improved safety and maturity of laparoscopic surgery. Considering the results from the questions on the conventional laparoscopic surgical system, surgeons showed a tendency to overcome current constraints regarding laparoscopic surgery with the AI-based surgical assistant system, especially with regard to the skillfulness of assistants.

    The majority of responders anticipated that the introduction of the AI-based surgical assistant system would replace the existing workforce by 41%-60%. Therefore, an AI-based surgical assistant system for AISS could be a great solution in university hospitals where resident working hours are regulated (eg, 80-hour resident special act in Korea and The European Working Time Directive in Europe) [18,20]. Of course, there may be some opinions concerning undertraining of residents, but the introduction of educational tools, such as simulation training systems, is a possible alternative [17,21].

    Issues Related to Practical Application

    Before adopting and introducing an AI-based surgical assistant system in the surgical field, ethical and legal responsibilities should be discussed through consensus of medical, legal, and administrative experts and others. Additionally, although not included in this survey, the recent development of AI is likely to include explainable AI, a concept contrasted with previous black-box AI, in the development of new technologies.

    At the time of the introduction of robotic surgery, which is being actively used presently, many experts had discussed ethical issues [22-24]. Current robotic surgery is a master-slave system, with the surgeon having most of the responsibility, making it easy to discuss ethical issues. However, in the case of an autonomous AI-based surgical assistant system, there may be controversy regarding the responsibility for harm and injuries caused to the patient during the robotic surgery, and social discussions about this need to be carried out for the adoption of an AI-based surgical assistant system [24,25].

    Explainability should be considered when newly developing AI-based surgical assistant systems. Current AI-based medical programs involving deep learning and machine learning techniques lack explainability, hindering the dependence of medical professionals on conclusions from these programs. Therefore, considering the characteristics of surgical procedures that are repeated continuously with small and large decisions, it is expected that explainability will be essential for the interaction between the machine and the operator and should be incorporated in the development of AI-based robotic assistance systems that contribute to these procedures [26].

    Strengths and Weaknesses

    This report is based on a survey among experts who have been actively performing laparoscopic surgery in various fields. To the best of our knowledge, this is the first report showing the clinical need for an AI-based surgical assistant system for AISS according to an e-survey. Moreover, this study is meaningful because we could identify the unmet need of clinicians for an AI-based system for AISS, which could be developed soon. However, this study has some limitations. First, it was challenging to check the exact response rate through the mailing system used in this study, which could act as a bias, and thus, the results of this study should be interpreted carefully. However, we could assume that the questionnaire was answered by our targeted responders because most of the responders mentioned that they performed more than one surgery per month. Second, the specific national health insurance system controlled by the government in the Republic of Korea could affect the expected value of an AI-based surgical assistant system for AISS, and the finding should be complemented by international surveys later. Third, the validity and reliability of the items in the questionnaire could not be confirmed because there has been no previous comparable study and this study targeted a specific group of experts in our country.

    Conclusion

    In the conventional laparoscopic surgical system, surgeons may value the proficiency of assistants, and most of them may feel uncomfortable with the unintended or not intuitive movement of laparoscopic cameras and devices. For the development of an AI-based surgical assistant system in the future, safe operation may be expected through lens cleaning, intuitive manipulation, and tissue damage minimization. Furthermore, an AI-based surgical assistant system is expected to replace approximately 41%-60% of the workforce, which may increase surgeons’ willingness to purchase such a system for reducing human resources and performing a comfortable, safe, and skilled operation. Conclusively, an AI-based surgical assistant system for AISS will become essential to enhance surgeons’ convenience, but it will be necessary to increase the safety and quality of surgery for patients.

    Acknowledgments

    This study was presented at the 27th Annual Meeting of the Korean Society of Gynecologic Endoscopy and Minimal Invasive Surgery and received the Best Oral Presentation Award. This research was supported by grants from Seoul National University (800-20170249, 800-20180201, and 800-20190437) and the National Research Foundation of Korea (2017057240).

    Authors' Contributions

    HSK designed this survey; HSK, EJL, SIK, SHK, and CWJ collected the data; and SJP performed the analysis and wrote the manuscript. All authors contributed to the revision of the manuscript.

    Conflicts of Interest

    None declared.

    References

    1. Miller DD, Brown EW. Artificial Intelligence in Medical Practice: The Question to the Answer? Am J Med 2018 Feb;131(2):129-133. [CrossRef] [Medline]
    2. Ramesh A, Kambhampati C, Monson J, Drew P. Artificial intelligence in medicine. Ann R Coll Surg Engl 2004 Sep 01;86(5):334-338 [FREE Full text] [CrossRef] [Medline]
    3. Mirnezami R, Ahmed A. Surgery 3.0, artificial intelligence and the next-generation surgeon. Br J Surg 2018 Apr;105(5):463-465. [CrossRef] [Medline]
    4. Perez RE, Schwaitzberg SD. Robotic surgery: finding value in 2019 and beyond. Ann. Laparosc. Endosc. Surg 2019 May;4:51-51. [CrossRef]
    5. El Hachem L, Momeni M, Friedman K, Moshier EL, Chuang LT, Gretz HF. Safety, feasibility and learning curve of robotic single-site surgery in gynecology. Int J Med Robot 2016 Sep 11;12(3):509-516. [CrossRef] [Medline]
    6. Thiel DD, Winfield HN. Robotics in urology: past, present, and future. J Endourol 2008 Apr;22(4):825-830. [CrossRef] [Medline]
    7. Sutherland GR, Wolfsberger S, Lama S, Zarei-nia K. The Evolution of neuroArm. Neurosurgery 2013;72:A27-A32. [CrossRef]
    8. van den Berg J, Miller S, Duckworth D, Hu H, Wan A. Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations. 2010 Presented at: 2010 IEEE International Conference on Robotics and Automation; May 3-8, 2010; Anchorage, Alaska. [CrossRef]
    9. Shademan A, Decker RS, Opfermann JD, Leonard S, Krieger A, Kim PC. Supervised autonomous robotic soft tissue surgery. Sci Transl Med 2016 May 04;8(337):337ra64-337ra64. [CrossRef] [Medline]
    10. Miehle J, Ostler D, Gerstenlauer N, Minker W. The next step: intelligent digital assistance for clinical operating rooms. Innov Surg Sci 2017 Sep;2(3):159-161 [FREE Full text] [CrossRef] [Medline]
    11. Mewes A, Hensen B, Wacker F, Hansen C. Touchless interaction with software in interventional radiology and surgery: a systematic literature review. Int J Comput Assist Radiol Surg 2017 Feb 19;12(2):291-305. [CrossRef] [Medline]
    12. Mentis H, O'Hara K, Gonzalez G, Sellen A, Corish R, Criminisi A. Voice or Gesture in the Operating Room. 2015 Apr Presented at: CHI '15: CHI Conference on Human Factors in Computing Systems; April, 2015; Seoul Republic of Korea. [CrossRef]
    13. Barbash GI, Glied SA. New Technology and Health Care Costs — The Case of Robot-Assisted Surgery. N Engl J Med 2010 Aug 19;363(8):701-704. [CrossRef]
    14. Kalantari DH, Kalantari DE, Maleki S. E-survey (surveys based on e-mail & web). Procedia Computer Science 2011;3:935-941. [CrossRef]
    15. nownsurvey.: ELIMNET Co, Ltd   URL: https://www.nownsurvey.com/ [accessed 2020-04-03]
    16. van den Haak L, Alleblas C, Nieboer TE, Rhemrev JP, Jansen FW. Efficacy and safety of uterine manipulators in laparoscopic surgery: a review. Arch Gynecol Obstet 2015 Nov 13;292(5):1003-1011. [CrossRef] [Medline]
    17. Kauvar DS, Braswell A, Brown BD, Harnisch M. Influence of resident and attending surgeon seniority on operative performance in laparoscopic cholecystectomy. J Surg Res 2006 May 15;132(2):159-163. [CrossRef] [Medline]
    18. Kim DJ, Kim SG. Comparative study of the operative experience of surgical residents before and after 80-hour work week restrictions. Ann Surg Treat Res 2018 Nov;95(5):233-239 [FREE Full text] [CrossRef] [Medline]
    19. Coverdill JE, Shelton JS, Alseidi A, Borgstrom DC, Dent DL, Dumire R, et al. The promise and problems of non-physician practitioners in general surgery education: Results of a multi-center, mixed-methods study of faculty. Am J Surg 2018 Feb;215(2):222-226. [CrossRef] [Medline]
    20. Fitzgerald J, Caesar B. The European Working Time Directive: a practical review for surgical trainees. Int J Surg 2012;10(8):399-403 [FREE Full text] [CrossRef] [Medline]
    21. Graafland M, Bok K, Schreuder HW, Schijven MP. A multicenter prospective cohort study on camera navigation training for key user groups in minimally invasive surgery. Surg Innov 2014 Jun 16;21(3):312-319. [CrossRef] [Medline]
    22. Larson JA, Johnson MH, Bhayani SB. Application of surgical safety standards to robotic surgery: five principles of ethics for nonmaleficence. J Am Coll Surg 2014 Feb;218(2):290-293. [CrossRef] [Medline]
    23. Siqueira-Batista R, Souza CR, Maia PM, Siqueira SL. Robotic Surgery: Bioethical Aspects. Arq Bras Cir Dig 2016 Dec;29(4):287-290 [FREE Full text] [CrossRef] [Medline]
    24. O'Sullivan S, Nevejans N, Allen C, Blyth A, Leonard S, Pagallo U, et al. Legal, regulatory, and ethical frameworks for development of standards in artificial intelligence (AI) and autonomous robotic surgery. Int J Med Robot 2019 Feb 09;15(1):e1968. [CrossRef] [Medline]
    25. O'Sullivan S, Leonard S, Holzinger A, Allen C, Battaglia F, Nevejans N, et al. Anatomy 101 for AI-driven robotics: Explanatory, ethical and legal frameworks for development of cadaveric skills training standards in autonomous robotic surgery/autopsy. Int J Med Robot 2019 May 30:e2020. [CrossRef] [Medline]
    26. Holzinger A, Langs G, Denk H, Zatloukal K, Müller H. Causability and explainability of artificial intelligence in medicine. Wiley Interdiscip Rev Data Min Knowl Discov 2019 Apr 02;9(4):e1312 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    AI: artificial intelligence
    AISS: artificial intelligence–based solo surgery


    Edited by G Eysenbach; submitted 31.12.19; peer-reviewed by A Holzinger, N Mohammadzadeh; comments to author 25.02.20; revised version received 10.03.20; accepted 12.03.20; published 15.05.20

    ©Soo Jin Park, Eun Ji Lee, Se Ik Kim, Seong-Ho Kong, Chang Wook Jeong, Hee Seung Kim. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 15.05.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.