The Karma system is currently undergoing maintenance (Monday, January 29, 2018).
The maintenance period has been extended to 8PM EST.

Karma Credits will not be available for redeeming during maintenance.

Journal Description

JMIR Medical Informatics (JMI, ISSN 2291-9694) (Editor-in-chief: Christian Lovis MD MPH FACMI) is a top-rated, tier A journal which focuses on clinical informatics, big data in health and health care, decision support for health professionals, electronic health records, ehealth infrastructures and implementation. It has a focus on applied, translational research, with a broad readership including clinicians, CIOs, engineers, industry and health informatics professionals.

Published by JMIR Publications, publisher of the Journal of Medical Internet Research (JMIR), the leading eHealth/mHealth journal (Impact Factor 2017: 4.671), JMIR Med Inform has a slightly different scope (emphasizing more on applications for clinicians and health professionals rather than consumers/citizens, which is the focus of JMIR), publishes even faster, and also allows papers which are more technical or more formative than what would be published in the Journal of Medical Internet Research.

JMIR Medical Informatics journal features a rapid and thorough peer-review process, professional copyediting, professional production of PDF, XHTML, and XML proofs (ready for deposit in PubMed Central/PubMed). The site is optimized for mobile and iPad use.

JMIR Medical Informatics adheres to the same quality standards as JMIR and all articles published here are also cross-listed in the Table of Contents of JMIR, the worlds' leading medical journal in health sciences / health services research and health informatics (


Recent Articles:

  • Source: Pixnio; Copyright: Pixnio; URL:; License: Public Domain (CC0).

    Utilization of Electronic Medical Records and Biomedical Literature to Support the Diagnosis of Rare Diseases Using Data Fusion and Collaborative Filtering...


    Background: In the United States, a rare disease is characterized as the one affecting no more than 200,000 patients at a certain period. Patients suffering from rare diseases are often either misdiagnosed or left undiagnosed, possibly due to insufficient knowledge or experience with the rare disease on the part of clinical practitioners. With an exponentially growing volume of electronically accessible medical data, a large volume of information on thousands of rare diseases and their potentially associated diagnostic information is buried in electronic medical records (EMRs) and medical literature. Objective: This study aimed to leverage information contained in heterogeneous datasets to assist rare disease diagnosis. Phenotypic information of patients existed in EMRs and biomedical literature could be fully leveraged to speed up diagnosis of diseases. Methods: In our previous work, we advanced the use of a collaborative filtering recommendation system to support rare disease diagnostic decision making based on phenotypes derived solely from EMR data. However, the influence of using heterogeneous data with collaborative filtering was not discussed, which is an essential problem while facing large volumes of data from various resources. In this study, to further investigate the performance of collaborative filtering on heterogeneous datasets, we studied EMR data generated at Mayo Clinic as well as published article abstracts retrieved from the Semantic MEDLINE Database. Specifically, in this study, we designed different data fusion strategies from heterogeneous resources and integrated them with the collaborative filtering model. Results: We evaluated performance of the proposed system using characterizations derived from various combinations of EMR data and literature, as well as with sole EMR data. We extracted nearly 13 million EMRs from the patient cohort generated between 2010 and 2015 at Mayo Clinic and retrieved all article abstracts from the semistructured Semantic MEDLINE Database that were published till the end of 2016. We applied a collaborative filtering model and compared the performance generated by different metrics. Log likelihood ratio similarity combined with k-nearest neighbor on heterogeneous datasets showed the optimal performance in patient recommendation with area under the precision-recall curve (PRAUC) 0.475 (string match), 0.511 (systematized nomenclature of medicine [SNOMED] match), and 0.752 (Genetic and Rare Diseases Information Center [GARD] match). Log likelihood ratio similarity also performed the best with mean average precision 0.465 (string match), 0.5 (SNOMED match), and 0.749 (GARD match). Performance of rare disease prediction was also demonstrated by using the optimal algorithm. Macro-average F-measure for string, SNOMED, and GARD match were 0.32, 0.42, and 0.63, respectively. Conclusions: This study demonstrated potential utilization of heterogeneous datasets in a collaborative filtering model to support rare disease diagnosis. In addition to phenotypic-based analysis, in the future, we plan to further resolve the heterogeneity issue and reduce miscommunication between EMR and literature by mining genotypic information to establish a comprehensive disease-phenotype-gene network for rare disease diagnosis.

  • A patient receiving instructions from their pharmacist. Source: Image created by the Authors; Copyright: The Authors; URL:; License: Creative Commons Attribution (CC-BY).

    Extraction and Standardization of Patient Complaints from Electronic Medication Histories for Pharmacovigilance: Natural Language Processing Analysis in...


    Background: Despite the growing number of studies using natural language processing for pharmacovigilance, there are few reports on manipulating free text patient information in Japanese. Objective: This study aimed to establish a method of extracting and standardizing patient complaints from electronic medication histories accumulated in a Japanese community pharmacy for the detection of possible adverse drug event (ADE) signals. Methods: Subjective information included in electronic medication history data provided by a Japanese pharmacy operating in Hiroshima, Japan from September 1, 2015 to August 31, 2016, was used as patients’ complaints. We formulated search rules based on morphological analysis and daily (nonmedical) speech and developed a system that automatically executes the search rules and annotates free text data with International Classification of Diseases, Tenth Revision (ICD-10) codes. The performance of the system was evaluated through comparisons with data manually annotated by health care workers for a data set of 5000 complaints. Results: Of 5000 complaints, the system annotated 2236 complaints with ICD-10 codes, whereas health care workers annotated 2348 statements. There was a match in the annotation of 1480 complaints between the system and manual work. System performance was .66 regarding precision, .63 in recall, and .65 for the F-measure. Conclusions: Our results suggest that the system may be helpful in extracting and standardizing patients’ speech related to symptoms from massive amounts of free text data, replacing manual work. After improving the extraction accuracy, we expect to utilize this system to detect signals of possible ADEs from patients’ complaints in the future.

  • Clinical Document Viewer showing common information types shared through an interoperable Electronic Health Record and an electrocardiogram (montage). Source: Tim Graham; Copyright: Tim Graham; URL:; License: Licensed by JMIR.

    Emergency Physician Use of the Alberta Netcare Portal, a Province-Wide Interoperable Electronic Health Record: Multi-Method Observational Study


    Background: The adoption and use of an electronic health record (EHR) can facilitate real-time access to key health information and support improved outcomes. Many Canadian provinces use interoperable EHRs (iEHRs) to facilitate health information exchange, but the clinical use and utility of iEHRs has not been well described. Objective: The aim of this study was to describe the use of a provincial iEHR known as the Alberta Netcare Portal (ANP) in 4 urban Alberta emergency departments. The secondary objectives were to characterize the time spent using the respective electronic tools and identify the aspects that were perceived as most useful by emergency department physicians. Methods: In this study, we have included 4 emergency departments, 2 using paper-based ordering (University of Alberta Hospital [UAH] and Grey Nuns Community Hospital [GNCH]) and 2 using a commercial vendor clinical information system (Peter Lougheed Centre [PLC] and Foothills Medical Centre [FMC]). Structured clinical observations of ANP use and system audit logs analysis were compared at the 4 sites from October 2014 to March 2016. Results: Observers followed 142 physicians for a total of 566 hours over 376 occasions. The median percentage of observed time spent using ANP was 8.5% at UAH (interquartile range, IQR, 3.7%-13.3%), 4.4% at GNCH (IQR 2.4%-4.4%), 4.6% at FMC (IQR 2.4%-7.6%), and 5.1% at PLC (IQR 3.0%-7.7%). By combining administrative and access audit data, the median number of ANP screens (ie, results and reports displayed on a screen) accessed per patient visit were 20 at UAH (IQR 6-67), 9 at GNCH (IQR 4-29), 7 at FMC (IQR 2-18), and 5 at PLC (IQR 2-14). When compared with the structured clinical observations, the statistical analysis of screen access data showed that ANP was used more at UAH than the other sites. Conclusions: This study shows that the iEHR is well utilized at the 4 sites studied, and the usage patterns implied clinical value. Use of the ANP was highest in a paper-based academic center and lower in the centers using a commercial emergency department clinical information system. More study about the clinical impacts of using iEHRs in the Canadian context including longer term impacts on quality of practice and safety are required.

  • Source: Freepik; Copyright: Freepik; URL:; License: Licensed by JMIR.

    Defining Empowerment and Supporting Engagement Using Patient Views From the Citizen Health Information Portal: Qualitative Study


    Background: The increasing presence of technology in health care has created new opportunities for patient engagement and with this, an intensified exploration of patient empowerment within the digital health context. While the use of technology, such as patient portals, has been positively received, a clear linkage between digital health solutions, patient empowerment, and health outcomes remains elusive. Objective: The primary objective of this research was to explore the views of participants enrolled in an electronic health record portal access trial regarding the resultant influence of this technology on their feelings of patient empowerment. Methods: The exploration of patient empowerment within a digital health context was done with participants in a tethered patient portal trial using interpretive description. Interpretive description is a qualitative methodology developed to pragmatically address clinical health questions. Patient demographics, self-reported health status, and self-identified technology adaptation contributed to the assessment of empowerment in this qualitative approach. Results: This research produced a view of patient empowerment within the digital health context summarized in two overarching categories: (1) Being Heard and (2) Moving Forward. In each of these, two subcategories further delineate the aspects of empowerment, as viewed by these participants: Knowing More and Seeing What They See under Being Heard, and Owning Future Steps and Promoting Future Care under Moving Forward. This work also highlighted an ongoing interconnectedness between the concepts of patient empowerment, engagement, and activation and the need to further articulate the unique aspects of each of these. Conclusions: The results of this study contribute needed patient voice to the ongoing evolution of the concept of patient empowerment. In order to move toward more concrete and accurate measure of patient empowerment and engagement in digital health, there must be further consideration of what patients themselves identify as essential aspects of these complex concepts. This research has revealed relational and informational elements as two key areas of focus in the ongoing evolution of patient empowerment operationalization and measure.

  • Source: iStock by Getty Images; Copyright: marchmeena29; URL:; License: Licensed by the authors.

    Validating a Framework for Coding Patient-Reported Health Information to the Medical Dictionary for Regulatory Activities Terminology: An Evaluative Study


    Background: The availability of and interest in patient-generated health data (PGHD) have grown steadily. Patients describe medical experiences differently compared with how clinicians or researchers would describe their observations of those same experiences. Patients may find nonserious, known adverse drug events (ADEs) to be an ongoing concern, which impacts the tolerability and adherence. Clinicians must be vigilant for medically serious, potentially fatal ADEs. Having both perspectives provides patients and clinicians with a complete picture of what to expect from drug therapies. Multiple initiatives seek to incorporate patients’ perspectives into drug development, including PGHD exploration for pharmacovigilance. The Food and Drug Administration (FDA) Adverse Event Reporting System contains case reports of postmarketing ADEs. To facilitate the analysis of these case reports, case details are coded using the Medical Dictionary for Regulatory Activities (MedDRA). PatientsLikeMe is a Web-based network where patients report, track, share, and discuss their health information. PatientsLikeMe captures PGHD through free-text and structured data fields. PatientsLikeMe structured data are coded to multiple medical terminologies, including MedDRA. The standardization of PatientsLikeMe PGHD enables electronic accessibility and enhances patient engagement. Objective: The aim of this study is to retrospectively review PGHD for symptoms and ADEs entered by patients on PatientsLikeMe and coded by PatientsLikeMe to MedDRA terminology for concordance with regulatory-focused coding practices. Methods: An FDA MedDRA coding expert retrospectively reviewed a data file containing verbatim patient-reported symptoms and ADEs and PatientsLikeMe-assigned MedDRA terms to determine the medical accuracy and appropriateness of the selected MedDRA terms, applying the International Council for Harmonisation MedDRA Term Selection: Points to Consider (MTS:PTC) guides. Results: The FDA MedDRA coding expert reviewed 3234 PatientsLikeMe-assigned MedDRA codes and patient-reported verbatim text. The FDA and PatientsLikeMe were concordant at 97.09% (3140/3234) of the PatientsLikeMe-assigned MedDRA codes. The 2.91% (94/3234) discordant subset was analyzed to identify reasons for differences. Coding differences were attributed to several reasons but mostly driven by PatientsLikeMe’s approach of assigning a more general MedDRA term to enable patient-to-patient engagement, while the FDA assigned a more specific medically relevant term. Conclusions: PatientsLikeMe MedDRA coding of PGHD was generally comparable to how the FDA would code similar data, applying the MTS:PTC principles. Discordant coding resulted from several reasons but mostly reflected a difference in purpose. The MTS:PTC coding principles aim to capture the most specific reported information about an ADE, whereas PatientsLikeMe may code patient-reported symptoms and ADEs to more general MedDRA terms to support patient engagement among a larger group of patients. This study demonstrates that most verbatim reports of symptoms and ADEs collected by a PGHD source, such as the PatientsLikeMe platform, could be reliably coded to MedDRA terminology by applying the MTS:PTC guide. Regarding all secondary use of novel data, understanding coding and standardization principles applied to these data types are important.

  • Three-dimensional portable document format (montage). Source: The Authors / Rawpixel; Copyright: Rawpixel; URL:; License: Licensed by the authors.

    Three-Dimensional Portable Document Format (3D PDF) in Clinical Communication and Biomedical Sciences: Systematic Review of Applications, Tools, and Protocols


    Background: The Portable Document Format (PDF) is the standard file format for the communication of biomedical information via the internet and for electronic scholarly publishing. Although PDF allows for the embedding of three-dimensional (3D) objects and although this technology has great potential for the communication of such data, it is not broadly used by the scientific community or by clinicians. Objective: The objective of this review was to provide an overview of existing publications that apply 3D PDF technology and the protocols and tools for the creation of model files and 3D PDFs for scholarly purposes to demonstrate the possibilities and the ways to use this technology. Methods: A systematic literature review was performed using PubMed and Google Scholar. Articles searched for were in English, peer-reviewed with biomedical reference, published since 2005 in a journal or presented at a conference or scientific meeting. Ineligible articles were removed after screening. The found literature was categorized into articles that (1) applied 3D PDF for visualization, (2) showed ways to use 3D PDF, and (3) provided tools or protocols for the creation of 3D PDFs or necessary models. Finally, the latter category was analyzed in detail to provide an overview of the state of the art. Results: The search retrieved a total of 902 items. Screening identified 200 in-scope publications, 13 covering the use of 3D PDF for medical purposes. Only one article described a clinical routine use case; all others were pure research articles. The disciplines that were covered beside medicine were many. In most cases, either animal or human anatomies were visualized. A method, protocol, software, library, or other tool for the creation of 3D PDFs or model files was described in 19 articles. Most of these tools required advanced programming skills and/or the installation of further software packages. Only one software application presented an all-in-one solution with a graphical user interface. Conclusions: The use of 3D PDF for visualization purposes in clinical communication and in biomedical publications is still not in common use, although both the necessary technique and suitable tools are available, and there are many arguments in favor of this technique. The potential of 3D PDF usage should be disseminated in the clinical and biomedical community. Furthermore, easy-to-use, standalone, and free-of-charge software tools for the creation of 3D PDFs should be developed.

  • Map of key stakeholders of the NERP step 1 with whom qualitative in-depth interviews were conducted (montage). Source: The Authors /; Copyright: JMIR Publications; URL:; License: Creative Commons Attribution (CC-BY).

    Implementing a National Electronic Referral Program: Qualitative Study


    Background: Electronic referrals or e-referrals can be defined as the electronic transmission of patient data and clinical requests between health service providers. National electronic referral systems have proved challenging to implement due to problems of fit between the technical systems proposed and the existing sociotechnical systems. In seeming contradiction to a sociotechnical approach, the Irish Health Service Executive initiated an incremental implementation of a National Electronic Referral Programme (NERP), with step 1 including only the technical capability for general practitioners to submit electronic referral requests to hospital outpatient departments. The technology component of the program was specified, but any changes required to embed that technology in the existing sociotechnical system were not specified. Objective: This study aimed to theoretically frame the lessons learned from the NERP step 1 on the design and implementation of a national health information technology program. Methods: A case study design was employed, using qualitative interviews with key stakeholders of the NERP step 1 (N=41). A theory-driven thematic analysis of the interview data was conducted, using Barker et al’s Framework for Going to Full Scale. Results: The NERP step 1 was broadly welcomed by key stakeholders as the first step in the implementation of electronic referrals—delivering improvements in the speed, completeness of demographic information, and legibility and traceability of referral requests. National leadership and digitalized health records in general practice were critical enabling factors. Inhibiting factors included policy uncertainty about the future organizational structures within which electronic referrals would be implemented; the need to establish a central referral office consistent with these organizational structures; outstanding interoperability issues between the electronic referral solution and hospital patient administration systems; and an anticipated need to develop specialist referral templates for some specialties. A lack of specification of the sociotechnical elements of the NERP step 1 inhibited the necessary testing and refinement of the change package used to implement the program. Conclusions: The key strengths of the NERP step 1 are patient safety benefits. The NERP was progressed beyond the pilot stage despite limited resources and outstanding interoperability issues. In addition, a new electronic health unit in Ireland (eHealth Ireland) gained credibility in delivering national health information technology programs. Limitations of the program are its poor integration in the wider policy and quality improvement agenda of the Health Service Executive. The lack of specification of the sociotechnical elements of the program created challenges in communicating the program scope to key stakeholders and restricted the ability of program managers and implementers to test and refine the change package. This study concludes that while the sociotechnical elements of a national health information technology program do not need to be specified in tandem with technical elements, they do need to be specified early in the implementation process so that the change package used to implement the program can be tested and refined.

  • Source: StockSnap; Copyright: William Stitt; URL:; License: Public Domain (CC0).

    Uncovering a Role for Electronic Personal Health Records in Reducing Disparities in Sexually Transmitted Infection Rates Among Students at a Predominantly...


    Background: Black youth continue to bear an overwhelming proportion of the United States sexually transmitted infection (STI) burden, including HIV. Several studies on web-based and mobile health (mHealth) STI interventions have focused on characterizing strategies to improve HIV-related prevention and treatment interventions, risk communication, and stigma among men who have sex with men (MSM), people who use substances, and adolescent populations. The Electronic Sexual Health Information Notification and Education (eSHINE) Study was an exploratory mixed-methods study among students at a historically black university exploring perceptions on facilitating STI testing conversations with partners using electronic personal health records (PHRs). Objective: The purpose of this paper is to use eSHINE Study results to describe perceived impacts of electronic PHRs on facilitating STI testing discussions between sexual partners. Methods: Semistructured focus groups and individual in-depth interviews were conducted on a heterogeneous sample of students (n=35) between May and July 2014. Qualitative phase findings guided development of an online survey instrument for quantitative phase data collection. Online surveys were conducted using a convenience sample of students (n=354) between January and May 2015. Online survey items collected demographic information, sexual behaviors, beliefs and practices surrounding STI testing communication between partners, and beliefs about the impact of electronic PHR access on facilitating these discussions with partners. Chi-square analysis was performed to assess gender differences across quantitative measures. A Wilcoxon signed rank sum test was used to test the null hypothesis that electronic PHRs are believed to have no effect on the timing of dyadic STI health communication. Results: Participants described multiple individual and dyadic-level factors that inhibit initiating discussions about STI testing and test results with partners. Electronic PHRs were believed to improve ability to initiate conversations and confidence in STI screening information shared by partners. Among online survey participants, men were more likely to believe electronic PHRs make it easier to facilitate STI talks with potential partners (59.9% vs 51.9%; χ2=3.93, P=.05). The Wilcoxon signed-rank test results indicate significant increases in perceived discussion timing before sex with electronic PHR access (61.0% vs 40.4%; P<.001). Conclusions: Findings suggest that electronic PHR access in STI screening settings among similar populations of Black youth may improve both motivation and personal agency for initiating dyadic STI health communication. Results from this study will likely inform novel interventions that use access to electronic PHRs to stimulate important health-related discussions between sexual partners. Moving forward requires studying strategies for implementing interventions that leverage electronic PHRs to create new sexual health communication channels with providers, peers, and family among black youth.

  • Source: Freepik; Copyright: pressfoto; URL:; License: Licensed by JMIR.

    Task-Data Taxonomy for Health Data Visualizations: Web-Based Survey With Experts and Older Adults


    Background: Increasingly, eHealth involves health data visualizations to enable users to better understand their health situation. Selecting efficient and ergonomic visualizations requires knowledge about the task that the user wants to carry out and the type of data to be displayed. Taxonomies of abstract tasks and data types bundle this knowledge in a general manner. Task-data taxonomies exist for visualization tasks and data. They also exist for eHealth tasks. However, there is currently no joint task taxonomy available for health data visualizations incorporating the perspective of the prospective users. One of the most prominent prospective user groups of eHealth are older adults, but their perspective is rarely considered when constructing tasks lists. Objective: The aim of this study was to construct a task-data taxonomy for health data visualizations based on the opinion of older adults as prospective users of eHealth systems. eHealth experts served as a control group against the bias of lacking background knowledge. The resulting taxonomy would then be used as an orientation in system requirement analysis and empirical evaluation and to facilitate a common understanding and language in eHealth data visualization. Methods: Answers from 98 participants (51 older adults and 47 eHealth experts) given in an online survey were quantitatively analyzed, compared between groups, and synthesized into a task-data taxonomy for health data visualizations. Results: Consultation, diagnosis, mentoring, and monitoring were confirmed as relevant abstract tasks in eHealth. Experts and older adults disagreed on the importance of mentoring (χ24=14.1, P=.002) and monitoring (χ24=22.1, P<.001). The answers to the open questions validated the findings from the closed questions and added therapy, communication, cooperation, and quality management to the aforementioned tasks. Here, group differences in normalized code counts were identified for “monitoring” between the expert group (mean 0.18, SD 0.23) and the group of older adults (mean 0.08, SD 0.15; t96=2431, P=.02). Time-dependent data was most relevant across all eHealth tasks. Finally, visualization tasks and data types were assigned to eHealth tasks by both experimental groups. Conclusions: We empirically developed a task-data taxonomy for health data visualizations with prospective users. This provides a general framework for theoretical concession and for the prioritization of user-centered system design and evaluation. At the same time, the functionality dimension of the taxonomy for telemedicine—chosen as the basis for the construction of present taxonomy—was confirmed.

  • Source: Flickr; Copyright: US Department of Agriculture; URL:; License: Creative Commons Attribution (CC-BY).

    Adverse Drug Event Reporting From Clinical Care: Mixed-Methods Analysis for a Minimum Required Dataset


    Background: Patients commonly transition between health care settings, requiring care providers to transfer medication utilization information. Yet, information sharing about adverse drug events (ADEs) remains nonstandardized. Objective: The objective of our study was to describe a minimum required dataset for clinicians to document and communicate ADEs to support clinical decision making and improve patient safety. Methods: We used mixed-methods analysis to design a minimum required dataset for ADE documentation and communication. First, we completed a systematic review of the existing ADE reporting systems. After synthesizing reporting concepts and data fields, we conducted fieldwork to inform the design of a preliminary reporting form. We presented this information to clinician end-user groups to establish a recommended dataset. Finally, we pilot-tested and refined the dataset in a paper-based format. Results: We evaluated a total of 1782 unique data fields identified in our systematic review that describe the reporter, patient, ADE, and suspect and concomitant drugs. Of these, clinicians requested that 26 data fields be integrated into the dataset. Avoiding the need to report information already available electronically, reliance on prospective rather than retrospective causality assessments, and omitting fields deemed irrelevant to clinical care were key considerations. Conclusions: By attending to the information needs of clinicians, we developed a standardized dataset for adverse drug event reporting. This dataset can be used to support communication between care providers and integrated into electronic systems to improve patient safety. If anonymized, these standardized data may be used for enhanced pharmacovigilance and research activities.

  • Source: Wikimedia Commons; Copyright: Kgbo; URL:; License: Creative Commons Attribution + ShareAlike (CC-BY-SA).

    Nurses’ Experience With Health Information Technology: Longitudinal Qualitative Study


    Background: Nurses are the largest group of health information technology (HIT) users. As such, nurses’ adaptations are critical for HIT implementation success. However, longitudinal approaches to understanding nurses’ perceptions of HIT remain underexplored. Previous studies of nurses’ perceptions demonstrate that the progress and timing for acceptance of and adaptation to HIT varies. Objective: This study aimed to explore nurses’ experience regarding implementation of HIT over time. Methods: A phenomenological approach was used for this longitudinal qualitative study to explore nurses’ perceptions of HIT implementation over time, focusing on three time points (rounds) at 3, 9, and 18 months after implementation of electronic health records and bar code medication administration. The purposive sample was comprised of clinical nurses who worked on a medical-surgical unit in an academic center. Results: Major findings were categorized into 7 main themes with 54 subthemes. Nurses reported personal-level and organizational-level factors that facilitated HIT adaptation. We also generated network graphs to illustrate the occurrence of themes. Thematic interconnectivity differed due to nurses’ concerns and satisfaction at different time points. Equipment and workflow were the most frequent themes across all three rounds. Nurses were the most dissatisfied approximately 9 months after HIT implementation. Eighteen months after HIT implementation, nurses’ perceptions appeared more balanced. Conclusions: It is recommended that organizations invest in equipment (ie, wireless barcode scanners), refine policies to reflect nursing practice, and improve systems to focus on patient safety. Future research is necessary to confirm patterns of nurses’ adaptation to HIT in other samples.

  • Preterm infant in the NICU. Source: Flickr; Copyright: The Hudson Family; URL:; License: Creative Commons Attribution (CC-BY).

    The Impact of Implementation of a Clinically Integrated Problem-Based Neonatal Electronic Health Record on Documentation Metrics, Provider Satisfaction, and...


    Background: A goal of effective electronic health record provider documentation platforms is to provide an efficient, concise, and comprehensive notation system that will effectively reflect the clinical course, including the diagnoses, treatments, and interventions. Objective: The aim is to fully redesign and standardize the provider documentation process, seeking improvement in documentation based on ongoing All Patient Refined Diagnosis Related Group–based coding records, while maintaining noninferiority comparing provider satisfaction to our existing documentation process. We estimated the fiscal impact of improved documentation based on changes in expected hospital payments. Methods: Employing a multidisciplinary collaborative approach, we created an integrated clinical platform that captures data entry from the obstetrical suite, delivery room, neonatal intensive care unit (NICU) nursing and respiratory therapy staff. It provided the sole source for hospital provider documentation in the form of a history and physical exam, daily progress notes, and discharge summary. Health maintenance information, follow-up appointments, and running contemporaneous updated hospital course information have selected shared entry and common viewing by the NICU team. The interventions were to (1) improve provider awareness of appropriate documentation through a provider education handout and follow-up group discussion and (2) fully redesign and standardize the provider documentation process building from the native Epic-based software. The measures were (1) hospital coding department review of all NICU admissions and 3M All Patient Refined Diagnosis Related Group–based calculations of severity of illness, risk of mortality, and case mix index scores; (2) balancing measure: provider time utilization case study and survey; and (3) average expected hospital payment based on acuity-based clinical logic algorithm and payer mix. Results: We compared preintervention (October 2015-October 2016) to postintervention (November 2016-May 2017) time periods and saw: (1) significant improvement in All Patient Refined Diagnosis Related Group–derived severity of illness, risk of mortality, and case mix index (monthly average severity of illness scores increased by 11.1%, P=.008; monthly average risk of mortality scores increased by 13.5%, P=.007; and monthly average case mix index scores increased by 7.7%, P=.009); (2) time study showed increased time to complete history and physical and progress notes and decreased time to complete discharge summary (history and physical exam: time allocation increased by 47%, P=.05; progress note: time allocation increased by 91%, P<.001; discharge summary: time allocation decreased by 41%, P=.03); (3) survey of all providers: overall there was positive provider perception of the new documentation process based on a survey of the provider group; (4) significantly increased hospital average expected payments: comparing the preintervention and postintervention study periods, there was a US $14,020 per month per patient increase in average expected payment for hospital charges (P<.001). There was no difference in payer mix during this time period. Conclusions: A problem-based NICU documentation electronic health record more effectively improves documentation without dissatisfaction by the participating providers and improves hospital estimations of All Patient Refined Diagnosis Related Group–based revenue.

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Latest Submissions Open for Peer-Review:

View All Open Peer Review Articles
  • Word embedding for French natural language in healthcare: a comparative study

    Date Submitted: Sep 25, 2018

    Open Peer Review Period: Sep 30, 2018 - Nov 25, 2018

    Background: Word embedding technologies are now used in a wide range of applications. However, no formal evaluation and comparison have been made on models produced by the three most famous implementa...

    Background: Word embedding technologies are now used in a wide range of applications. However, no formal evaluation and comparison have been made on models produced by the three most famous implementations (Word2Vec, GloVe and FastText). Objective: The goal of this study is to compare embedding implementations on a corpus of documents produced in a working context, by health professionals. Methods: Models have been trained on documents coming from the Rouen university hospital. This data is not structured and cover a wide range of documents produced in a clinic (discharge summary, prescriptions ...). Four evaluation tasks have been defined (cosine similarity, odd one, mathematical operations and human formal evaluation) and applied on each model. Results: Word2Vec had the highest score for three of the four tasks (mathematical operations, odd one similarity and human validation), particularly regarding the Skip-Gram architecture. Conclusions: Even if this implementation had the best rate, each model has its own qualities and defects, like the training time which is very short for GloVe or morphosyntaxic similarity conservation observed with FastText. Models and test sets produced by this study will be the first publicly available through a graphical interface to help advance French biomedical research.

  • Classification of excluded or missing virology data from UK-wide influenza vaccine effectiveness studies

    Date Submitted: Sep 11, 2018

    Open Peer Review Period: Sep 15, 2018 - Nov 10, 2018

    Background: Monitoring the effectiveness of the influenza vaccination programme within the UK is necessary in order to assess its clinical impact. Data are collected from general practice sentinel net...

    Background: Monitoring the effectiveness of the influenza vaccination programme within the UK is necessary in order to assess its clinical impact. Data are collected from general practice sentinel network computerised medical record (CMR) systems on patients from whom virology specimens have been taken for influenza. The data collected includes demographics, comorbidities, vaccine exposure and if patients have had a virology specimen taken. Unfortunately not all virology specimens collected can be used in the vaccine effectiveness (VE) studies conducted. Objective: To describe the proportion, reasons and any trends in virology specimen data collected but not used in influenza VE analyses, with the goal of defining strategies to reduce collection of specimens ineligible for use in VE studies. Methods: We examined UK influenza VE studies from the past 10 years and identified incidences where data were labelled unusable. We categorised reasons for not using data as: (1) Vaccination history: Missing or Uncertain categories (including patient not registered with the practice at the start of the season); (2) Swab timing: Not recorded; More than 7 days (historically over 29 days) after symptom onset or within 14 days of vaccination; (3) Laboratory: Not sufficient data for processing (e.g. no age), CT values; (4) Flu or vaccination type of no interest (including pandemic years). The proportion, reasons and trends for data loss were identified through descriptive statistics and graphical representations. We included an analysis of where other data had been available at the point of analysis but not used. Results: Over 30% (13292/41337) of virology specimen data was not used across all seasons. Data loss gradually began to decrease from 2014/15 onwards. Data loss were highest for flu or vaccination type of no interest and swab timing. Retrospective and prospective actions were identified to reduce data loss in future. Around 60% of samples could have been included if identifiable data were better shared between records. Conclusions: The reasons for excluding samples and missing data varied, particularly prior to 2014; consistent categorisation was in place from 2014 onwards. Leaving aside the different issues around pandemic years, many of the virology swabs not included were due to suboptimal case selection by practices, but over half (58%) could have been included if identifiable data were better shared between data sources. Clinical Trial: N/A

  • Anesthesiology Control Tower: Feasibility Assessment to Support Translation (ACTFAST) a mixed methods study of a novel telemedicine-based support system for the operating room

    Date Submitted: Sep 9, 2018

    Open Peer Review Period: Sep 13, 2018 - Nov 8, 2018

    Background: Major postoperative morbidity and mortality remain common despite efforts to improve patient outcomes. Health information technologies, such as decision support systems, have the potential...

    Background: Major postoperative morbidity and mortality remain common despite efforts to improve patient outcomes. Health information technologies, such as decision support systems, have the potential to advance the standard of perioperative patient care. Failure to evaluate the usability of these technologies and barriers to their implementation can limit their acceptance within health systems. Objective: This manuscript describes the usability and acceptability of and systematic process for developing and adapting an innovative telemedicine based clinical support system, the Anesthesiology Control Tower. It also reports stakeholders’ perceptions of the barriers and facilitators the implementation of the intervention. Methods: Three phases of testing were conducted in an iterative manner in order to evaluate both the individual components of the Anesthesiology Control Tower and their integration as a whole. Phase 1 testing employed a “think-aloud” protocol analysis to identify surface level usability problems with individual software components of the ACT, in addition to the entirety of the structure. Phase 2 testing involved an extended qualitative and quantitative in-situ usability analysis. Phase 3 sought to identify major barriers and facilitators to implementation of the ACT through semi-structured interviews with key stakeholders. Results: Numerous usability problems with the software components of the ACT were identified in the Phase 1 and Phase 2 usability testing sessions. In response to these problems, seven iterations of the ACT software platform were developed. Initial satisfaction with the ACT, as measured by standardized measures, was below commonly accepted cutoffs for these measures. Satisfaction improved to acceptable levels over the course of revision and testing. A number of barriers to implementation were identified and addressed during the refinement of the ACT intervention. Conclusions: The Anesthesiology Control Tower system has the potential to improve the standard of perioperative anesthesia care. Through our thorough and iterative usability testing process and stakeholder assessment of barriers and facilitators, we were able to maximize the acceptability of this novel technology, thus improving our ability to implement this innovation into the model of care for perioperative medicine.

  • Lessons Learned from Implementing a Heart Failure Telemonitoring System in Home Care Nursing

    Date Submitted: Aug 21, 2018

    Open Peer Review Period: Aug 26, 2018 - Oct 21, 2018

    Background: Telemonitoring (TM) of heart failure patients in a clinic setting has been shown to be effective if properly implemented, but little is known about the feasibility and impact of implementi...

    Background: Telemonitoring (TM) of heart failure patients in a clinic setting has been shown to be effective if properly implemented, but little is known about the feasibility and impact of implementing TM through a home care nursing agency. Objective: The goal of this study was to determine the feasibility of implementing a smartphone-based TM system through a home care nursing agency, and to explore the feasibility of conducting a future effectiveness trial. Methods: A feasibility study was conducted, aiming to recruit 10-15 heart failure patients who would use the TM system for 4 months by taking daily measurements of weight and blood pressure, and recording symptoms. Home care nurses responded to alerts generated by the TM system either through a phone call and/or home visit. Results: Only six patients were recruited over a six-month period due to lack of referrals from physicians. Potential benefits of TM through a home care nursing agency were indicated, including through improved patient education, providing nurses with a better understanding of the patient’s health status, and reductions in home visits. Barriers to implementation included challenges in nurses contacting patients and physicians, retention issues, and integrating the TM system into a complex home care nursing workflow. Conclusions: Lessons learned included the need to incentivize physicians, to ensure streamlined processes for recruitment and communication, to target appropriate patient populations, and to create a core clinical group. Barriers encountered in this feasibility trial should be considered to determine their applicability when deploying innovations into different service delivery models.

  • Opportunities and Challenges of Telehealth in Remote Communities: A Case Study of the Yukon Telehealth System

    Date Submitted: Aug 20, 2018

    Open Peer Review Period: Aug 24, 2018 - Oct 19, 2018

    Background: Telehealth has been shown to improve access to healthcare and to reduce costs to the patient and healthcare system, especially for patients living in rural settings. However, unique challe...

    Background: Telehealth has been shown to improve access to healthcare and to reduce costs to the patient and healthcare system, especially for patients living in rural settings. However, unique challenges arise when implementing telehealth in remote communities. Objective: The objectives of this evaluation were to understand the current use, challenges, and opportunities of the Yukon Telehealth System. The lessons learned from this case study were used to determine important factors to consider when attempting to advance and expand telehealth programs in remote communities. Methods: A mixed-methods approach was used to evaluate the Yukon Telehealth System and to determine possible future advances. Quantitative data were obtained through usage logs. In addition, online questionnaires were administered to nurses in each of the 14 Yukon community health centres outside of Whitehorse, and patients who had used telehealth were also asked to complete a questionnaire. Qualitative data included focus groups and semi-structured interviews with a total of 36 telehealth stakeholders. Results: Since 2008, there have been a consistent total number of telehealth sessions of about 1000 per year, with the main use being for clinical care (70% of all sessions in 2015). From the questionnaire data (11 community nurses, 10 patients) and interview data, there was a consensus among the clinicians and patients that the System provided timely access and cost savings from reduced travel. However, they believed that it was underutilized and the equipment was outdated. The challenges and opportunities discovered led to the identification of four factors that should be considered when trying to advance and expand a telehealth program. 1) Patient and clinician buy-in: Past telehealth experiences should be considered when advancing the system, such as negative clinician experiences with outdated technology. Expansion of services in orthopaedics, dermatology, and psychiatry were found to have particular benefit in Yukon by clinicians specializing in these areas. 2) Workflow: The use and scheduling of telehealth should be streamlined and automated as much as possible to reduce dependencies on the single Yukon Telehealth Coordinator. 3) Access to telehealth technology: Clinicians and patients should have easy access to telehealth technology, whether it is telehealth units or alternative desktop applications. The use of consumer products, such as mobile technology, should be leveraged as appropriate. 4) Infrastructure: The required human resources and technology need to be established when expanding and advancing telehealth. Conclusions: While clinicians and patients have generally positive perceptions of the Yukon Telehealth System, there was consensus that it was underutilized. Many opportunities exist to significantly expand the types of telehealth services and the number of telehealth sessions. The lessons learned from this evaluation can be applied to other remote communities to realize telehealth’s potential as a means for efficient, safe, convenient, and cost-effective care delivery.