JMIR Publications

The Karma system is currently undergoing maintenance (Monday, January 29, 2018).
The maintenance period has been extended to 8PM EST.

Karma Credits will not be available for redeeming during maintenance.

JMIR Medical Informatics

Clinical informatics, decision support for health professionals, electronic health records, and ehealth infrastructures.


Journal Description

JMIR Medical Informatics (JMI, ISSN 2291-9694) is a top-rated, tier A journal which focuses on clinical informatics, big data in health and health care, decision support for health professionals, electronic health records, ehealth infrastructures and implementation. It has a focus on applied, translational research, with a broad readership including clinicians, CIOs, engineers, industry and health informatics professionals.

Published by JMIR Publications, publisher of the Journal of Medical Internet Research (JMIR), the leading eHealth/mHealth journal (Impact Factor 2016: 5.175), JMIR Med Inform has a slightly different scope (emphasizing more on applications for clinicians and health professionals rather than consumers/citizens, which is the focus of JMIR), publishes even faster, and also allows papers which are more technical or more formative than what would be published in the Journal of Medical Internet Research.

JMIR Medical Informatics journal features a rapid and thorough peer-review process, professional copyediting, professional production of PDF, XHTML, and XML proofs (ready for deposit in PubMed Central/PubMed). The site is optimized for mobile and iPad use.

JMIR Medical Informatics adheres to the same quality standards as JMIR and all articles published here are also cross-listed in the Table of Contents of JMIR, the worlds' leading medical journal in health sciences / health services research and health informatics (


Recent Articles:

  • Source: Pixabay; Copyright: Free-Photos; URL:; License: Public Domain (CC0).

    Characterizing and Managing Missing Structured Data in Electronic Health Records: Data Analysis


    Background: Missing data is a challenge for all studies; however, this is especially true for electronic health record (EHR)-based analyses. Failure to appropriately consider missing data can lead to biased results. While there has been extensive theoretical work on imputation, and many sophisticated methods are now available, it remains quite challenging for researchers to implement these methods appropriately. Here, we provide detailed procedures for when and how to conduct imputation of EHR laboratory results. Objective: The objective of this study was to demonstrate how the mechanism of missingness can be assessed, evaluate the performance of a variety of imputation methods, and describe some of the most frequent problems that can be encountered. Methods: We analyzed clinical laboratory measures from 602,366 patients in the EHR of Geisinger Health System in Pennsylvania, USA. Using these data, we constructed a representative set of complete cases and assessed the performance of 12 different imputation methods for missing data that was simulated based on 4 mechanisms of missingness (missing completely at random, missing not at random, missing at random, and real data modelling). Results: Our results showed that several methods, including variations of Multivariate Imputation by Chained Equations (MICE) and softImpute, consistently imputed missing values with low error; however, only a subset of the MICE methods was suitable for multiple imputation. Conclusions: The analyses we describe provide an outline of considerations for dealing with missing EHR data, steps that researchers can perform to characterize missingness within their own data, and an evaluation of methods that can be applied to impute clinical data. While the performance of methods may vary between datasets, the process we describe can be generalized to the majority of structured data types that exist in EHRs, and all of our methods and code are publicly available.

  • Source: Flickr; Copyright: US Food and Drug Administration; URL:; License: Public Domain (CC0).

    Representation of Time-Relevant Common Data Elements in the Cancer Data Standards Repository: Statistical Evaluation of an Ontological Approach


    Background: Today, there is an increasing need to centralize and standardize electronic health data within clinical research as the volume of data continues to balloon. Domain-specific common data elements (CDEs) are emerging as a standard approach to clinical research data capturing and reporting. Recent efforts to standardize clinical study CDEs have been of great benefit in facilitating data integration and data sharing. The importance of the temporal dimension of clinical research studies has been well recognized; however, very few studies have focused on the formal representation of temporal constraints and temporal relationships within clinical research data in the biomedical research community. In particular, temporal information can be extremely powerful to enable high-quality cancer research. Objective: The objective of the study was to develop and evaluate an ontological approach to represent the temporal aspects of cancer study CDEs. Methods: We used CDEs recorded in the National Cancer Institute (NCI) Cancer Data Standards Repository (caDSR) and created a CDE parser to extract time-relevant CDEs from the caDSR. Using the Web Ontology Language (OWL)–based Time Event Ontology (TEO), we manually derived representative patterns to semantically model the temporal components of the CDEs using an observing set of randomly selected time-related CDEs (n=600) to create a set of TEO ontological representation patterns. In evaluating TEO’s ability to represent the temporal components of the CDEs, this set of representation patterns was tested against two test sets of randomly selected time-related CDEs (n=425). Results: It was found that 94.2% (801/850) of the CDEs in the test sets could be represented by the TEO representation patterns. Conclusions: In conclusion, TEO is a good ontological model for representing the temporal components of the CDEs recorded in caDSR. Our representative model can harness the Semantic Web reasoning and inferencing functionalities and present a means for temporal CDEs to be machine-readable, streamlining meaningful searches.

  • A Jefferson Health doctor in your medicine cabinet. Source: Image created by Authors; Copyright: The Authors; URL:; License: Creative Commons Attribution (CC-BY).

    Patient and Health System Experience With Implementation of an Enterprise-Wide Telehealth Scheduled Video Visit Program: Mixed-Methods Study


    Background: Real-time video visits are increasingly used to provide care in a number of settings because they increase access and convenience of care, yet there are few reports of health system experiences. Objective: The objective of this study is to report health system and patient experiences with implementation of a telehealth scheduled video visit program across a health system. Methods: This is a mixed methods study including (1) a retrospective descriptive report of implementation of a telehealth scheduled visit program at one large urban academic-affiliated health system and (2) a survey of patients who participated in scheduled telehealth visits. Health system and patient-reported survey measures were aligned with the National Quality Forum telehealth measure reporting domains of access, experience, and effectiveness of care. Results: This study describes implementation of a scheduled synchronous video visit program over an 18-month period. A total of 3018 scheduled video visits were completed across multiple clinical departments. Patient experiences were captured in surveys of 764 patients who participated in telehealth visits. Among survey respondents, 91.6% (728/795) reported satisfaction with the scheduled visits and 82.7% (628/759) reported perceived quality similar to an in-person visit. A total of 86.0% (652/758) responded that use of the scheduled video visit made it easier to get care. Nearly half (46.7%, 346/740) of patients estimated saving 1 to 3 hours and 40.8% (302/740) reported saving more than 3 hours of time. The net promoter score, a measure of patient satisfaction, was very high at 52. Conclusions: A large urban multihospital health system implemented an enterprise-wide scheduled telehealth video visit program across a range of clinical specialties with a positive patient experience. Patients found use of scheduled video visits made it easier to get care and the majority perceived time saved, suggesting that use of telehealth for scheduled visits can improve potential access to care across a range of clinical scenarios with favorable patient experiences.

  • Source:; Copyright: David Castillo Dominici; URL:; License: Licensed by the authors.

    The Use of Communication Apps by Medical Staff in the Australian Health Care System: Survey Study on Prevalence and Use


    Background: The use of communication apps on mobile phones offers an efficient, unobtrusive, and portable mode of communication for medical staff. The potential enhancements in patient care and education appear significant, with clinical details able to be shared quickly within multidisciplinary teams, supporting rapid integration of disparate information, and more efficient patient care. However, sharing patient data in this way also raises legal and ethical issues. No data is currently available demonstrating how widespread the use of these apps are, doctor’s attitudes towards them, or what guides clinician choice of app. Objective: The objective of this study was to quantify and qualify the use of communication apps among medical staff in clinical situations, their role in patient care, and knowledge and attitudes towards safety, key benefits, potential disadvantages, and policy implications. Methods: Medical staff in hospitals across Victoria (Australia) were invited to participate in an anonymous 33-question survey. The survey collected data on respondent’s demographics, their use of communication apps in clinical settings, attitudes towards communication apps, perceptions of data “safety,” and why one communication app was chosen over others. Results: Communication apps in Victorian hospitals are in widespread use from students to consultants, with WhatsApp being the primary app used. The median number of messages shared per day was 12, encompassing a range of patient information. All respondents viewed these apps positively in quickly communicating patient information in a clinical setting; however, all had concerns about the privacy implications arising from sharing patient information in this way. In total, 67% (60/90) considered patient data “moderately safe” on these apps, and 50% (46/90) were concerned the use of these apps was inconsistent with current legislation and policy. Apps were more likely to be used if they were fast, easy to use, had an easy login process, and were already in widespread use. Conclusions: Communication app use by medical personnel in Victorian hospitals is pervasive. These apps contribute to enhanced communication between medical staff, but their use raises compliance issues, most notably with Australian privacy legislation. Development of privacy-compliant apps such as MedX needs to prioritize a user-friendly interface and market the product as a privacy-compliant comparator to apps previously adapted to health care settings.

  • Source: Pixnio; Copyright: Pixnio; URL:; License: Public Domain (CC0).

    Automated Information Extraction on Treatment and Prognosis for Non–Small Cell Lung Cancer Radiotherapy Patients: Clinical Study


    Background: In outcome studies of oncology patients undergoing radiation, researchers extract valuable information from medical records generated before, during, and after radiotherapy visits, such as survival data, toxicities, and complications. Clinical studies rely heavily on these data to correlate the treatment regimen with the prognosis to develop evidence-based radiation therapy paradigms. These data are available mainly in forms of narrative texts or table formats with heterogeneous vocabularies. Manual extraction of the related information from these data can be time consuming and labor intensive, which is not ideal for large studies. Objective: The objective of this study was to adapt the interactive information extraction platform Information and Data Extraction using Adaptive Learning (IDEAL-X) to extract treatment and prognosis data for patients with locally advanced or inoperable non–small cell lung cancer (NSCLC). Methods: We transformed patient treatment and prognosis documents into normalized structured forms using the IDEAL-X system for easy data navigation. The adaptive learning and user-customized controlled toxicity vocabularies were applied to extract categorized treatment and prognosis data, so as to generate structured output. Results: In total, we extracted data from 261 treatment and prognosis documents relating to 50 patients, with overall precision and recall more than 93% and 83%, respectively. For toxicity information extractions, which are important to study patient posttreatment side effects and quality of life, the precision and recall achieved 95.7% and 94.5% respectively. Conclusions: The IDEAL-X system is capable of extracting study data regarding NSCLC chemoradiation patients with significant accuracy and effectiveness, and therefore can be used in large-scale radiotherapy clinical data studies.

  • Source:; Copyright: Marcy Sanchez; URL:; License: Public Domain (CC0).

    Quality of Decision Support in Computerized Provider Order Entry: Systematic Literature Review


    Background: Computerized decision support systems have raised a lot of hopes and expectations in the field of order entry. Although there are numerous studies reporting positive impacts, concerns are increasingly high about alert fatigue and effective impacts of these systems. One of the root causes of fatigue alert reported is the low clinical relevance of these alerts. Objective: The objective of this systematic review was to assess the reported positive predictive value (PPV), as a proxy to clinical relevance, of decision support systems in computerized provider order entry (CPOE). Methods: A systematic search of the scientific literature published between February 2009 and March 2015 on CPOE, clinical decision support systems, and the predictive value associated with alert fatigue was conducted using PubMed database. Inclusion criteria were as follows: English language, full text available (free or pay for access), assessed medication, direct or indirect level of predictive value, sensitivity, or specificity. When possible with the information provided, PPV was calculated or evaluated. Results: Additive queries on PubMed retrieved 928 candidate papers. Of these, 376 were eligible based on abstract. Finally, 26 studies qualified for a full-text review, and 17 provided enough information for the study objectives. An additional 4 papers were added from the references of the reviewed papers. The results demonstrate massive variations in PPVs ranging from 8% to 83% according to the object of the decision support, with most results between 20% and 40%. The best results were observed when patients’ characteristics, such as comorbidity or laboratory test results, were taken into account. There was also an important variation in sensitivity, ranging from 38% to 91%. Conclusions: There is increasing reporting of alerts override in CPOE decision support. Several causes are discussed in the literature, the most important one being the clinical relevance of alerts. In this paper, we tried to assess formally the clinical relevance of alerts, using a near-strong proxy, which is the PPV of alerts, or any way to express it such as the rate of true and false positive alerts. In doing this literature review, three inferences were drawn. First, very few papers report direct or enough indirect elements that support the use or the computation of PPV, which is a gold standard for all diagnostic tools in medicine and should be systematically reported for decision support. Second, the PPV varies a lot according to the typology of decision support, so that overall rates are not useful, but must be reported by the type of alert. Finally, in general, the PPVs are below or near 50%, which can be considered as very low.

  • Source:; Copyright: hamiltonpaviana; URL:; License: Public Domain (CC0).

    A Clinical Decision Support Engine Based on a National Medication Repository for the Detection of Potential Duplicate Medications: Design and Evaluation


    Background: A computerized physician order entry (CPOE) system combined with a clinical decision support system can reduce duplication of medications and thus adverse drug reactions. However, without infrastructure that supports patients’ integrated medication history across health care facilities nationwide, duplication of medication can still occur. In Taiwan, the National Health Insurance Administration has implemented a national medication repository and Web-based query system known as the PharmaCloud, which allows physicians to access their patients’ medication records prescribed by different health care facilities across Taiwan. Objective: This study aimed to develop a scalable, flexible, and thematic design-based clinical decision support (CDS) engine, which integrates a national medication repository to support CPOE systems in the detection of potential duplication of medication across health care facilities, as well as to analyze its impact on clinical encounters. Methods: A CDS engine was developed that can download patients’ up-to-date medication history from the PharmaCloud and support a CPOE system in the detection of potential duplicate medications. When prescribing a medication order using the CPOE system, a physician receives an alert if there is a potential duplicate medication. To investigate the impact of the CDS engine on clinical encounters in outpatient services, a clinical encounter log was created to collect information about time, prescribed drugs, and physicians’ responses to handling the alerts for each encounter. Results: The CDS engine was installed in a teaching affiliate hospital, and the clinical encounter log collected information for 3 months, during which a total of 178,300 prescriptions were prescribed in the outpatient departments. In all, 43,844/178,300 (24.59%) patients signed the PharmaCloud consent form allowing their physicians to access their medication history in the PharmaCloud. The rate of duplicate medication was 5.83% (1843/31,614) of prescriptions. When prescribing using the CDS engine, the median encounter time was 4.3 (IQR 2.3-7.3) min, longer than that without using the CDS engine (median 3.6, IQR 2.0-6.3 min). From the physicians’ responses, we found that 42.06% (1908/4536) of the potential duplicate medications were recognized by the physicians and the medication orders were canceled. Conclusions: The CDS engine could easily extend functions for detection of adverse drug reactions when more and more electronic health record systems are adopted. Moreover, the CDS engine can retrieve more updated and completed medication histories in the PharmaCloud, so it can have better performance for detection of duplicate medications. Although our CDS engine approach could enhance medication safety, it would make for a longer encounter time. This problem can be mitigated by careful evaluation of adopted solutions for implementation of the CDS engine. The successful key component of a CDS engine is the completeness of the patient’s medication history, thus further research to assess the factors in increasing the PharmaCloud consent rate is required.

  • Source: Pexels; Copyright:; URL:; License: Public Domain (CC0).

    The Use of Technology in Identifying Hospital Malnutrition: Scoping Review


    Background: Malnutrition is a condition most commonly arising from the inadequate consumption of nutrients necessary to maintain physiological health and is associated with the development of cardiovascular disease, osteoporosis, and sarcopenia. Malnutrition occurring in the hospital setting is caused by insufficient monitoring, identification, and assessment efforts. Furthermore, the ability of health care workers to identify and recognize malnourished patients is suboptimal. Therefore, interventions focusing on the identification and treatment of malnutrition are valuable, as they reduce the risks and rates of malnutrition within hospitals. Technology may be a particularly useful ally in identifying malnutrition due to scalability, timeliness, and effectiveness. In an effort to explore the issue, this scoping review synthesized the availability of technological tools to detect and identify hospital malnutrition. Objective: Our objective was to conduct a scoping review of the different forms of technology used in addressing malnutrition among adults admitted to hospital to (1) identify the extent of the published literature on this topic, (2) describe key findings, and (3) identify outcomes. Methods: We designed and implemented a search strategy in 3 databases (PubMed, Scopus, and CINAHL). We completed a descriptive numerical summary and analyzed study characteristics. One reviewer independently extracted data from the databases. Results: We retrieved and reviewed a total of 21 articles. We categorized articles by the computerized tool or app type: malnutrition assessment (n=15), food intake monitoring (n=5), or both (n=1). Within those categories, we subcategorized the different technologies as either hardware (n=4), software (n=13), or both (n=4). An additional subcategory under software was cloud-based apps (n=1). Malnutrition in the acute hospital setting was largely an unrecognized problem, owing to insufficient monitoring, identification, and initial assessments of identifying both patients who are already malnourished and those who are at risk of malnourishment. Studies went on to examine the effectiveness of health care workers (nurses and doctors) with a knowledge base focused on clinical care and their ability to accurately and consistently identify malnourished geriatric patients within that setting. Conclusions: Most articles reported effectiveness in accurately increasing malnutrition detection and awareness. Computerized tools and apps may also help reduce health care workers’ workload and time spent assessing patients for malnutrition. Hospitals may also benefit from implementing malnutrition technology through observing decreased length of stay, along with decreased foregone costs related to missing malnutrition diagnoses. It is beneficial to study the impact of these technologies to examine possible areas of improvement. A future systematic review would further contribute to the evidence and effectiveness of the use of technologies in assessing and monitoring hospital malnutrition.

  • Source:; Copyright: Veterans Health; URL:; License: Public Domain (CC0).

    Automating Quality Measures for Heart Failure Using Natural Language Processing: A Descriptive Study in the Department of Veterans Affairs


    Background: We developed an accurate, stakeholder-informed, automated, natural language processing (NLP) system to measure the quality of heart failure (HF) inpatient care, and explored the potential for adoption of this system within an integrated health care system. Objective: To accurately automate a United States Department of Veterans Affairs (VA) quality measure for inpatients with HF. Methods: We automated the HF quality measure Congestive Heart Failure Inpatient Measure 19 (CHI19) that identifies whether a given patient has left ventricular ejection fraction (LVEF) <40%, and if so, whether an angiotensin-converting enzyme inhibitor or angiotensin-receptor blocker was prescribed at discharge if there were no contraindications. We used documents from 1083 unique inpatients from eight VA medical centers to develop a reference standard (RS) to train (n=314) and test (n=769) the Congestive Heart Failure Information Extraction Framework (CHIEF). We also conducted semi-structured interviews (n=15) for stakeholder feedback on implementation of the CHIEF. Results: The CHIEF classified each hospitalization in the test set with a sensitivity (SN) of 98.9% and positive predictive value of 98.7%, compared with an RS and SN of 98.5% for available External Peer Review Program assessments. Of the 1083 patients available for the NLP system, the CHIEF evaluated and classified 100% of cases. Stakeholders identified potential implementation facilitators and clinical uses of the CHIEF. Conclusions: The CHIEF provided complete data for all patients in the cohort and could potentially improve the efficiency, timeliness, and utility of HF quality measurements.

  • Source:; Copyright: tmeier1964; URL:; License: Public Domain (CC0).

    A Pilot Study of Biomedical Text Comprehension using an Attention-Based Deep Neural Reader: Design and Experimental Analysis


    Background: With the development of artificial intelligence (AI) technology centered on deep-learning, the computer has evolved to a point where it can read a given text and answer a question based on the context of the text. Such a specific task is known as the task of machine comprehension. Existing machine comprehension tasks mostly use datasets of general texts, such as news articles or elementary school-level storybooks. However, no attempt has been made to determine whether an up-to-date deep learning-based machine comprehension model can also process scientific literature containing expert-level knowledge, especially in the biomedical domain. Objective: This study aims to investigate whether a machine comprehension model can process biomedical articles as well as general texts. Since there is no dataset for the biomedical literature comprehension task, our work includes generating a large-scale question answering dataset using PubMed and manually evaluating the generated dataset. Methods: We present an attention-based deep neural model tailored to the biomedical domain. To further enhance the performance of our model, we used a pretrained word vector and biomedical entity type embedding. We also developed an ensemble method of combining the results of several independent models to reduce the variance of the answers from the models. Results: The experimental results showed that our proposed deep neural network model outperformed the baseline model by more than 7% on the new dataset. We also evaluated human performance on the new dataset. The human evaluation result showed that our deep neural model outperformed humans in comprehension by 22% on average. Conclusions: In this work, we introduced a new task of machine comprehension in the biomedical domain using a deep neural model. Since there was no large-scale dataset for training deep neural models in the biomedical domain, we created the new cloze-style datasets Biomedical Knowledge Comprehension Title (BMKC_T) and Biomedical Knowledge Comprehension Last Sentence (BMKC_LS) (together referred to as BioMedical Knowledge Comprehension) using the PubMed corpus. The experimental results showed that the performance of our model is much higher than that of humans. We observed that our model performed consistently better regardless of the degree of difficulty of a text, whereas humans have difficulty when performing biomedical literature comprehension tasks that require expert level knowledge.

  • Source:; Copyright: Pina Messina; URL:; License: Public Domain (CC0).

    Stage-Based Mobile Intervention for Substance Use Disorders in Primary Care: Development and Test of Acceptability


    Background: In 2016, 21 million Americans aged 12 years and older needed treatment for a substance use disorder (SUD). However, only 10% to 11% of individuals requiring SUD treatment received it. Given their access to patients, primary care providers are in a unique position to perform universal Screening, Brief Intervention, and Referral to Treatment (SBIRT) to identify individuals at risk, fill gaps in services, and make referrals to specialty treatment when indicated. Major barriers to SBIRT include limited time among providers and low motivation to change among many patients. Objective: The objective of this study was to develop and test the acceptability of a prototype of a mobile-delivered substance use risk intervention (SURI) for primary care patients and a clinical dashboard for providers that can address major barriers to SBIRT for risky drug use. The SURI delivers screening and feedback on SUD risk via mobile tools to patients at home or in the waiting room; for patients at risk, it also delivers a brief intervention based on the transtheoretical model of behavior change (TTM) to facilitate progress through the stages of change for quitting the most problematic drug and for seeking treatment if indicated. The prototype also delivers 30 days of stage-matched text messages and 4 Web-based activities addressing key topics. For providers, the clinical dashboard summarizes the patient’s SUD risk scores and stage of change data, and provides stage-matched scripts to guide in-person sessions. Methods: A total of 4 providers from 2 federally qualified health centers (FQHCs) were recruited for the pilot test, and they in turn recruited 5 patients with a known SUD. Furthermore, 3 providers delivered dashboard-guided SBIRT sessions and completed a brief acceptability survey. A total of 4 patients completed a Web-based SURI session and in-person SBIRT session, accessed other program components, and completed 3 acceptability surveys over 30 days. Questions in the surveys were adapted from the National Cancer Institute’s Education Materials Review Form. Response options ranged from 1=strongly disagree to 5=strongly agree. The criterion for establishing acceptability was an overall rating of 4.0 or higher across items. Results: For providers, the overall mean acceptability rating was 4.4 (standard deviation [SD] 0.4). Notably, all providers gave a rating of 5.0 for the item, “The program can give me helpful information about my patient.” For patients, the overall mean acceptability rating was 4.5 (SD 0.3) for the mobile- and provider-delivered SBIRT sessions and 4.0 (SD 0.4) for the text messages and Web-based activities. One highly rated item was “The program could help me make some positive changes” (4.5). Conclusions: The SURI program and clinical dashboard, developed to reduce barriers to SBIRT in primary care, were well received by providers and patients.

  • Radiology CT technician and patient being scanned and diagnosed on CT. Source: iStock by Getty Images; Copyright: John Kellerman; URL:; License: Licensed by the authors.

    Standard Anatomic Terminologies: Comparison for Use in a Health Information Exchange–Based Prior Computed Tomography (CT) Alerting System


    Background: A health information exchange (HIE)–based prior computed tomography (CT) alerting system may reduce avoidable CT imaging by notifying ordering clinicians of prior relevant studies when a study is ordered. For maximal effectiveness, a system would alert not only for prior same CTs (exams mapped to the same code from an exam name terminology) but also for similar CTs (exams mapped to different exam name terminology codes but in the same anatomic region) and anatomically proximate CTs (exams in adjacent anatomic regions). Notification of previous same studies across an HIE requires mapping of local site CT codes to a standard terminology for exam names (such as Logical Observation Identifiers Names and Codes [LOINC]) to show that two studies with different local codes and descriptions are equivalent. Notifying of prior similar or proximate CTs requires an additional mapping of exam codes to anatomic regions, ideally coded by an anatomic terminology. Several anatomic terminologies exist, but no prior studies have evaluated how well they would support an alerting use case. Objective: The aim of this study was to evaluate the fitness of five existing standard anatomic terminologies to support similar or proximate alerts of an HIE-based prior CT alerting system. Methods: We compared five standard anatomic terminologies (Foundational Model of Anatomy, Systematized Nomenclature of Medicine Clinical Terms, RadLex, LOINC, and LOINC/Radiological Society of North America [RSNA] Radiology Playbook) to an anatomic framework created specifically for our use case (Simple ANatomic Ontology for Proximity or Similarity [SANOPS]), to determine whether the existing terminologies could support our use case without modification. On the basis of an assessment of optimal terminology features for our purpose, we developed an ordinal anatomic terminology utility classification. We mapped samples of 100 random and the 100 most frequent LOINC CT codes to anatomic regions in each terminology, assigned utility classes for each mapping, and statistically compared each terminology’s utility class rankings. We also constructed seven hypothetical alerting scenarios to illustrate the terminologies’ differences. Results: Both RadLex and the LOINC/RSNA Radiology Playbook anatomic terminologies ranked significantly better (P<.001) than the other standard terminologies for the 100 most frequent CTs, but no terminology ranked significantly better than any other for 100 random CTs. Hypothetical scenarios illustrated instances where no standard terminology would support appropriate proximate or similar alerts, without modification. Conclusions: LOINC/RSNA Radiology Playbook and RadLex’s anatomic terminologies appear well suited to support proximate or similar alerts for commonly ordered CTs, but for less commonly ordered tests, modification of the existing terminologies with concepts and relations from SANOPS would likely be required. Our findings suggest SANOPS may serve as a framework for enhancing anatomic terminologies in support of other similar use cases.

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Latest Submissions Open for Peer-Review:

View All Open Peer Review Articles
  • Emergency physician use and perceptions of the Alberta Netcare Portal, a province-wide interoperable electronic health record

    Date Submitted: Feb 20, 2018

    Open Peer Review Period: Feb 22, 2018 - Apr 19, 2018

    Background: The adoption and use of an Electronic Health Record can facilitate real-time access to key health information and support improved outcomes. Many Canadian provinces use interoperable Elect...

    Background: The adoption and use of an Electronic Health Record can facilitate real-time access to key health information and support improved outcomes. Many Canadian provinces use interoperable Electronic Health Records (iEHRs) to facilitate Health Information Exchange (HIE), but, to date, the clinical use and utility of iEHRs has not been well-described. Objective: Our study's primary objective was to describe the use and reported utility of a provincial iEHR known as the Alberta Netcare Portal (ANP) in four urban Alberta emergency departments (EDs). The secondary objectives were to characterize the time spent using the respective electronic tools, and which aspects were perceived as most useful by ED physicians. Methods: Four EDs were included in the study, two using paper-based ordering (University of Alberta Hospital [UAH] and Grey Nuns Community Hospital [GNCH]) and two using a commercial vendor Clinical Information System (Peter Lougheed Centre [PLC] and Foothills Medical Centre [FMC]). Structured clinical observations of ANP use, semi-structured interviews, and system audit logs analysis were compared at the four sites from October 2014 to March 2016. Results: Observers followed 142 physicians for a total of 566 hours over 376 occasions. The median percentage of observed time spent using ANP was 8.5% at UAH (interquartile range IQR: 3.7% - 13.3%), 4.4% at GNCH (2.4%-4.4%), 4.6% at FMC (2.4%-7.6%), and 5.1% at PLC (3.0%-7.7%). By combining administrative and access audit data, the median number of ANP screens (i.e., results and reports displayed on a screen) accessed per patient visit were 20 at UAH (IQR: 6-67), 9 at GNCH (4-29), 7 at FMC (2-18) and 5 at PLC (2-14) indicating that clinicians found significant value in using ANP while providing ED care. To explore this hypothesis, semi-structured interviews were analyzed using an inductive approach. The themes that emerged from the interviews were that the ANP improved the quality and continuity of care and patient safety. Further enhancements related to medication management would support better outcomes for patients. Conclusions: This study shows that the iEHR is well utilized at the four sites studied and physicians participating in the study perceived ANP had a positive impact on knowledge of their patients, patient safety, and quality and continuity of care. Physicians described high utility and usability of ANP. More study about the clinical impacts of using iEHRs in the Canadian context, including longer term impacts on quality of practice and safety are required.

  • Utilizing Electronic Health Records for Clinical Research: A Pilot to Build and Test Silent Best Practice Alert (BPA) Notifications for Patient Recruitment in Clinical Research

    Date Submitted: Feb 5, 2018

    Open Peer Review Period: Feb 5, 2018 - Apr 2, 2018

    Background: Participant recruitment, especially for frail elderly hospitalized patients, remains one of the greatest challenges for many research groups. Traditional recruitment methods such as chart...

    Background: Participant recruitment, especially for frail elderly hospitalized patients, remains one of the greatest challenges for many research groups. Traditional recruitment methods such as chart reviews or word of mouth notifications for patients in the inpatient setting are often inefficient, low-yielding, time consuming and expensive. Silent Best Practice Alert (BPA) systems have previously been used to improve clinical care but not in clinical research. Objective: This pilot project examined a new EPIC BPA system developed to identify potentially eligible participants in real time to help research teams maximize recruitment accuracy and efficiency of resources. We hypothesized that this tool would reduce the daily screening time, the number of missed potential participants as well as the overall cost needed to recruit the targeted number of patients. Methods: The BPA system was jointly developed by a clinical research and electronic medical records implementation/management team at Partners Healthcare. The was developed and pilot tested in an observational clinical trial to enroll patients admitted for acute exacerbation of chronic pulmonary disease (COPD). We compared the BPA system with our usual method of patient identification (chart reviews and word of mouth referrals) and evaluated for daily screening time, number of missed potential participants as well as the overall cost needed to recruit the targeted number of patients. Results: 559 potentially eligible patients were identified through the two screening methods compared. Of those, 460 patients were identified by both methods, with 99 found by just the Epic Workbench Method and 42 identified by just the silent BPA method. Of the 99 identified by the Epic Workbench, only 12 (12.12%) were considered eligible. Of the 42 identified by the silent BPA method, 30 (71.43%) were considered eligible. A total of 319 “Eligible” patients were identified, and of those 60 participants enrolled in the Emerald-COPD Study. Since implementation, the silent BPA system has found an equivalent of 3 additional patients per week. From the comparison, the silent BPA screening method was shown to be approximately 4 times (23.58%) faster than our previous screening method, projected to save 442.5 hours over the duration of the study. Conclusions: Automation of the recruitment process has allowed us to identify potential participants in real time and avoid missing patients. Silent BPA screening is a considerably faster method which allows for more efficient use of resources. This innovative and instrumental functionality can be specified to the needs of other research studies hoping to utilize the electronic medical records system for participant recruitment.

  • The construction principles, approaches, design considerations, and representation challenges of an ontology-based knowledge base prototype: OntoKBCF

    Date Submitted: Jan 29, 2018

    Open Peer Review Period: Jan 30, 2018 - Mar 27, 2018

    Ontology is a key enabling technology for the Semantic Web. Web Ontology Language (OWL) is the semantic markup language for publishing and sharing data via ontologies on the Web. OntoKBCF is an ontolo...

    Ontology is a key enabling technology for the Semantic Web. Web Ontology Language (OWL) is the semantic markup language for publishing and sharing data via ontologies on the Web. OntoKBCF is an ontology-based knowledge base prototype built in OWL to supply customizable molecular genetics information and health information about cystic fibrosis via EHR interfaces. This paper introduces the construction principles, approaches, design considerations, and representation challenges we faced in the construction of OntoKBCF. More specifically, we examine: (1) what is included in OntoKBCF; (2) how we organized and represented complicated knowledge facts by utilizing basic atomic concepts in a formal and machine-processable manner; (3) how the knowledge facts (i.e., known facts with straightforward or complicated statements) can be made automatically usable via an electronic health record system prototype; and 4) why we constructed OntoKBCF in this way. The main challenges include representing: (1) patient groups comprehensively; (2) uncertain knowledge, ambiguous concepts, and negative statements; and (3) more complicated and detailed molecular mechanisms or pathway information about cystic fibrosis. Although cystic fibrosis is utilized as an example, OntoKBCF should be able to be expanded in a straightforward manner based on its current structure. The construction principles can be referenced for building other human monogenetic diseases knowledge bases.

  • Validation of a Computerized Method for Measuring CTPA Yield in the Emergency Department

    Date Submitted: Jan 29, 2018

    Open Peer Review Period: Jan 30, 2018 - Mar 27, 2018

    Background: Use of computed tomography pulmonary angiography (CTPA) in the assessment for pulmonary embolism (PE) has significantly increased over the past two decades. While this technology has impro...

    Background: Use of computed tomography pulmonary angiography (CTPA) in the assessment for pulmonary embolism (PE) has significantly increased over the past two decades. While this technology has improved the accuracy of radiologic testing for PE, CTPA also carries the risk of significant iatrogenic harm. Each CTPA carries a 14% risk of contrast induced nephropathy and a lifetime malignancy risk that can be as high as 2.76%. Appropriate use of CTPA can be estimated by monitoring CTPA yield, the percentage of tests positive for PE. This is the first study to propose and validate a computerized method for measuring CTPA yield in the ED. Objective: To assess the validity of a novel computerized method of calculating CTPA yield in the ED. Methods: The electronic health record (EHR) databases at two tertiary care academic hospitals were queried for CTPA orders completed in the ED over one month periods. These visits were linked with an inpatient admission with a discharge diagnosis of PE based on International Classification of Diseases (ICD) codes. The computerized CTPA yield was calculated as the number of CTPA orders with an associated inpatient discharge diagnosis of PE divided by the total number of orders for completed CTPA. This computerized method was then validated by two independent reviewers performing a manual chart review, which included reading the free text radiology reports for each CTPA. Results: A total of 349 CTPA orders were completed during the one month periods at the two institutions. Acute PE was diagnosed on CTPA in 28 of these studies, with a CTPA yield of 7.7%. The computerized method correctly identified 27 of 28 scans positive for PE. The one discordant scan was tied to a patient who was discharged directly from the ED and as a result never received an inpatient discharge diagnosis. Conclusions: This is the first successful validation study of a computerized method for calculating CTPA yield in the ED. This method for data extraction allows for an accurate determination of CTPA yield and is more efficient than manual chart review. With this ability, healthcare systems can monitor for appropriate use of CTPA and the effect of interventions to reduce overuse and decrease preventable iatrogenic harm.

  • The Impact of Implementation of a Clinically-Integrated Problem-based Neonatal Electronic Medical Record- Documentation Metrics, Provider Satisfaction and Hospital Reimbursement. A Quality Improvement Project

    Date Submitted: Jan 8, 2018

    Open Peer Review Period: Jan 10, 2018 - Mar 7, 2018

    Background: A goal of effective EMR provider documentation platforms is to provide an efficient, concise and comprehensive notation system that will effectively reflect the clinical course, including...

    Background: A goal of effective EMR provider documentation platforms is to provide an efficient, concise and comprehensive notation system that will effectively reflect the clinical course, including the diagnoses, treatments and interventions. Objective: Fully redesign and standardize the provider documentation process, seeking improvement in documentation based upon ongoing APR-DRG-based coding records, while maintaining non-inferiority comparing provider satisfaction to our existing documentation process. We estimated the fiscal impact of improved documentation based upon changes in expected hospital payments. Methods: Employing a multidisciplinary collaborative approach, we created an integrated clinical platform that captures data entry from the obstetrical suite, delivery room, NICU nursing and respiratory therapy staff. It provides the sole source for hospital provider documentation in the form of a history and physical exam, daily progress notes, and discharge summary. Health maintenance information, follow-up appointments and running contemporaneous updated hospital course information have selected shared entry and common viewing by the NICU team. Interventions: (1) Improve provider awareness of appropriate documentation through a provider education hand-out and follow-up group discussion. (2) Fully redesign and standardize the provider documentation process building from the native Epic-based software. Measures: (1) Hospital coding department review of all NICU admissions and 3M APR-DRG based calculations of Severity of illness (SOI), risk of mortality (ROM) and case mix index (CMI) scores. (2) Balancing measure: Provider time utilization case study and survey; (3) Average expected hospital payment based upon acuity-based clinical logic algorithm and Payor mix. Results: We compared Pre-intervention (October 2015-October 2016) to Post-intervention (November 2016-May 2017) time-periods and demonstrated: (1) Significant improvement in APR-DRG derived SOI, ROM, CMI: Monthly average SOI scores increased by 11.1% (p = 0.008); Monthly average ROM scores increased by 13.5% ((p = 0.007); Monthly average CMI scores increased by 7.7% (p=0.009). (2) Time study showed increased time to complete H&P and progress notes and decreased time to complete discharge summary: H&P: time allocation increased by 47% (p = 0.053); Progress Note time allocation significantly increased by 91% (p < 0.001); Discharge summary time allocation significantly decreased by 41% (p = 0.032). (3) Survey of all providers: Overall there was positive provider perception of the new documentation process based upon a survey of the provider group. (4) Significantly increased hospital average expected payments: Comparing the PRE- and POST-intervention study periods, there was a $14,020/ month/ patient increase in Average Expected Payment for hospital charges (p < 0.001). There was no difference in payer mix during this time-period. Conclusions: A problem-based NICU documentation EMR more effectively improves documentation, without dissatisfaction by the participating providers, and improves hospital estimations of APR-DRG-based revenue.