The Karma system is currently undergoing maintenance (Monday, January 29, 2018).
The maintenance period has been extended to 8PM EST.

Karma Credits will not be available for redeeming during maintenance.

Journal Description

JMIR Medical Informatics (JMI, ISSN 2291-9694) (Editor-in-chief: Christian Lovis MD MPH FACMI) is a top-rated, tier A journal which focuses on clinical informatics, big data in health and health care, decision support for health professionals, electronic health records, ehealth infrastructures and implementation. It has a focus on applied, translational research, with a broad readership including clinicians, CIOs, engineers, industry and health informatics professionals.

Published by JMIR Publications, publisher of the Journal of Medical Internet Research (JMIR), the leading eHealth/mHealth journal (Impact Factor 2017: 4.671), JMIR Med Inform has a slightly different scope (emphasizing more on applications for clinicians and health professionals rather than consumers/citizens, which is the focus of JMIR), publishes even faster, and also allows papers which are more technical or more formative than what would be published in the Journal of Medical Internet Research.

JMIR Medical Informatics journal features a rapid and thorough peer-review process, professional copyediting, professional production of PDF, XHTML, and XML proofs (ready for deposit in PubMed Central/PubMed). The site is optimized for mobile and iPad use.

JMIR Medical Informatics adheres to the same quality standards as JMIR and all articles published here are also cross-listed in the Table of Contents of JMIR, the worlds' leading medical journal in health sciences / health services research and health informatics (


Recent Articles:

  • Clinical Document Viewer showing common information types shared through an interoperable Electronic Health Record and an electrocardiogram (montage). Source: Tim Graham; Copyright: Tim Graham; URL:; License: Licensed by JMIR.

    Emergency Physician Use of the Alberta Netcare Portal, a Province-Wide Interoperable Electronic Health Record: Multi-Method Observational Study


    Background: The adoption and use of an electronic health record (EHR) can facilitate real-time access to key health information and support improved outcomes. Many Canadian provinces use interoperable EHRs (iEHRs) to facilitate health information exchange, but the clinical use and utility of iEHRs has not been well described. Objective: The aim of this study was to describe the use of a provincial iEHR known as the Alberta Netcare Portal (ANP) in 4 urban Alberta emergency departments. The secondary objectives were to characterize the time spent using the respective electronic tools and identify the aspects that were perceived as most useful by emergency department physicians. Methods: In this study, we have included 4 emergency departments, 2 using paper-based ordering (University of Alberta Hospital [UAH] and Grey Nuns Community Hospital [GNCH]) and 2 using a commercial vendor clinical information system (Peter Lougheed Centre [PLC] and Foothills Medical Centre [FMC]). Structured clinical observations of ANP use and system audit logs analysis were compared at the 4 sites from October 2014 to March 2016. Results: Observers followed 142 physicians for a total of 566 hours over 376 occasions. The median percentage of observed time spent using ANP was 8.5% at UAH (interquartile range, IQR, 3.7%-13.3%), 4.4% at GNCH (IQR 2.4%-4.4%), 4.6% at FMC (IQR 2.4%-7.6%), and 5.1% at PLC (IQR 3.0%-7.7%). By combining administrative and access audit data, the median number of ANP screens (ie, results and reports displayed on a screen) accessed per patient visit were 20 at UAH (IQR 6-67), 9 at GNCH (IQR 4-29), 7 at FMC (IQR 2-18), and 5 at PLC (IQR 2-14). When compared with the structured clinical observations, the statistical analysis of screen access data showed that ANP was used more at UAH than the other sites. Conclusions: This study shows that the iEHR is well utilized at the 4 sites studied, and the usage patterns implied clinical value. Use of the ANP was highest in a paper-based academic center and lower in the centers using a commercial emergency department clinical information system. More study about the clinical impacts of using iEHRs in the Canadian context including longer term impacts on quality of practice and safety are required.

  • Source: Freepik; Copyright: Freepik; URL:; License: Licensed by JMIR.

    Defining Empowerment and Supporting Engagement Using Patient Views From the Citizen Health Information Portal: Qualitative Study


    Background: The increasing presence of technology in health care has created new opportunities for patient engagement and with this, an intensified exploration of patient empowerment within the digital health context. While the use of technology, such as patient portals, has been positively received, a clear linkage between digital health solutions, patient empowerment, and health outcomes remains elusive. Objective: The primary objective of this research was to explore the views of participants enrolled in an electronic health record portal access trial regarding the resultant influence of this technology on their feelings of patient empowerment. Methods: The exploration of patient empowerment within a digital health context was done with participants in a tethered patient portal trial using interpretive description. Interpretive description is a qualitative methodology developed to pragmatically address clinical health questions. Patient demographics, self-reported health status, and self-identified technology adaptation contributed to the assessment of empowerment in this qualitative approach. Results: This research produced a view of patient empowerment within the digital health context summarized in two overarching categories: (1) Being Heard and (2) Moving Forward. In each of these, two subcategories further delineate the aspects of empowerment, as viewed by these participants: Knowing More and Seeing What They See under Being Heard, and Owning Future Steps and Promoting Future Care under Moving Forward. This work also highlighted an ongoing interconnectedness between the concepts of patient empowerment, engagement, and activation and the need to further articulate the unique aspects of each of these. Conclusions: The results of this study contribute needed patient voice to the ongoing evolution of the concept of patient empowerment. In order to move toward more concrete and accurate measure of patient empowerment and engagement in digital health, there must be further consideration of what patients themselves identify as essential aspects of these complex concepts. This research has revealed relational and informational elements as two key areas of focus in the ongoing evolution of patient empowerment operationalization and measure.

  • Source: iStock by Getty Images; Copyright: marchmeena29; URL:; License: Licensed by the authors.

    Validating a Framework for Coding Patient-Reported Health Information to the Medical Dictionary for Regulatory Activities Terminology: An Evaluative Study


    Background: The availability of and interest in patient-generated health data (PGHD) have grown steadily. Patients describe medical experiences differently compared with how clinicians or researchers would describe their observations of those same experiences. Patients may find nonserious, known adverse drug events (ADEs) to be an ongoing concern, which impacts the tolerability and adherence. Clinicians must be vigilant for medically serious, potentially fatal ADEs. Having both perspectives provides patients and clinicians with a complete picture of what to expect from drug therapies. Multiple initiatives seek to incorporate patients’ perspectives into drug development, including PGHD exploration for pharmacovigilance. The Food and Drug Administration (FDA) Adverse Event Reporting System contains case reports of postmarketing ADEs. To facilitate the analysis of these case reports, case details are coded using the Medical Dictionary for Regulatory Activities (MedDRA). PatientsLikeMe is a Web-based network where patients report, track, share, and discuss their health information. PatientsLikeMe captures PGHD through free-text and structured data fields. PatientsLikeMe structured data are coded to multiple medical terminologies, including MedDRA. The standardization of PatientsLikeMe PGHD enables electronic accessibility and enhances patient engagement. Objective: The aim of this study is to retrospectively review PGHD for symptoms and ADEs entered by patients on PatientsLikeMe and coded by PatientsLikeMe to MedDRA terminology for concordance with regulatory-focused coding practices. Methods: An FDA MedDRA coding expert retrospectively reviewed a data file containing verbatim patient-reported symptoms and ADEs and PatientsLikeMe-assigned MedDRA terms to determine the medical accuracy and appropriateness of the selected MedDRA terms, applying the International Council for Harmonisation MedDRA Term Selection: Points to Consider (MTS:PTC) guides. Results: The FDA MedDRA coding expert reviewed 3234 PatientsLikeMe-assigned MedDRA codes and patient-reported verbatim text. The FDA and PatientsLikeMe were concordant at 97.09% (3140/3234) of the PatientsLikeMe-assigned MedDRA codes. The 2.91% (94/3234) discordant subset was analyzed to identify reasons for differences. Coding differences were attributed to several reasons but mostly driven by PatientsLikeMe’s approach of assigning a more general MedDRA term to enable patient-to-patient engagement, while the FDA assigned a more specific medically relevant term. Conclusions: PatientsLikeMe MedDRA coding of PGHD was generally comparable to how the FDA would code similar data, applying the MTS:PTC principles. Discordant coding resulted from several reasons but mostly reflected a difference in purpose. The MTS:PTC coding principles aim to capture the most specific reported information about an ADE, whereas PatientsLikeMe may code patient-reported symptoms and ADEs to more general MedDRA terms to support patient engagement among a larger group of patients. This study demonstrates that most verbatim reports of symptoms and ADEs collected by a PGHD source, such as the PatientsLikeMe platform, could be reliably coded to MedDRA terminology by applying the MTS:PTC guide. Regarding all secondary use of novel data, understanding coding and standardization principles applied to these data types are important.

  • Three-dimensional portable document format (montage). Source: The Authors / Rawpixel; Copyright: Rawpixel; URL:; License: Licensed by the authors.

    Three-Dimensional Portable Document Format (3D PDF) in Clinical Communication and Biomedical Sciences: Systematic Review of Applications, Tools, and Protocols


    Background: The Portable Document Format (PDF) is the standard file format for the communication of biomedical information via the internet and for electronic scholarly publishing. Although PDF allows for the embedding of three-dimensional (3D) objects and although this technology has great potential for the communication of such data, it is not broadly used by the scientific community or by clinicians. Objective: The objective of this review was to provide an overview of existing publications that apply 3D PDF technology and the protocols and tools for the creation of model files and 3D PDFs for scholarly purposes to demonstrate the possibilities and the ways to use this technology. Methods: A systematic literature review was performed using PubMed and Google Scholar. Articles searched for were in English, peer-reviewed with biomedical reference, published since 2005 in a journal or presented at a conference or scientific meeting. Ineligible articles were removed after screening. The found literature was categorized into articles that (1) applied 3D PDF for visualization, (2) showed ways to use 3D PDF, and (3) provided tools or protocols for the creation of 3D PDFs or necessary models. Finally, the latter category was analyzed in detail to provide an overview of the state of the art. Results: The search retrieved a total of 902 items. Screening identified 200 in-scope publications, 13 covering the use of 3D PDF for medical purposes. Only one article described a clinical routine use case; all others were pure research articles. The disciplines that were covered beside medicine were many. In most cases, either animal or human anatomies were visualized. A method, protocol, software, library, or other tool for the creation of 3D PDFs or model files was described in 19 articles. Most of these tools required advanced programming skills and/or the installation of further software packages. Only one software application presented an all-in-one solution with a graphical user interface. Conclusions: The use of 3D PDF for visualization purposes in clinical communication and in biomedical publications is still not in common use, although both the necessary technique and suitable tools are available, and there are many arguments in favor of this technique. The potential of 3D PDF usage should be disseminated in the clinical and biomedical community. Furthermore, easy-to-use, standalone, and free-of-charge software tools for the creation of 3D PDFs should be developed.

  • Map of key stakeholders of the NERP step 1 with whom qualitative in-depth interviews were conducted (montage). Source: The Authors /; Copyright: JMIR Publications; URL:; License: Creative Commons Attribution (CC-BY).

    Implementing a National Electronic Referral Program: Qualitative Study


    Background: Electronic referrals or e-referrals can be defined as the electronic transmission of patient data and clinical requests between health service providers. National electronic referral systems have proved challenging to implement due to problems of fit between the technical systems proposed and the existing sociotechnical systems. In seeming contradiction to a sociotechnical approach, the Irish Health Service Executive initiated an incremental implementation of a National Electronic Referral Programme (NERP), with step 1 including only the technical capability for general practitioners to submit electronic referral requests to hospital outpatient departments. The technology component of the program was specified, but any changes required to embed that technology in the existing sociotechnical system were not specified. Objective: This study aimed to theoretically frame the lessons learned from the NERP step 1 on the design and implementation of a national health information technology program. Methods: A case study design was employed, using qualitative interviews with key stakeholders of the NERP step 1 (N=41). A theory-driven thematic analysis of the interview data was conducted, using Barker et al’s Framework for Going to Full Scale. Results: The NERP step 1 was broadly welcomed by key stakeholders as the first step in the implementation of electronic referrals—delivering improvements in the speed, completeness of demographic information, and legibility and traceability of referral requests. National leadership and digitalized health records in general practice were critical enabling factors. Inhibiting factors included policy uncertainty about the future organizational structures within which electronic referrals would be implemented; the need to establish a central referral office consistent with these organizational structures; outstanding interoperability issues between the electronic referral solution and hospital patient administration systems; and an anticipated need to develop specialist referral templates for some specialties. A lack of specification of the sociotechnical elements of the NERP step 1 inhibited the necessary testing and refinement of the change package used to implement the program. Conclusions: The key strengths of the NERP step 1 are patient safety benefits. The NERP was progressed beyond the pilot stage despite limited resources and outstanding interoperability issues. In addition, a new electronic health unit in Ireland (eHealth Ireland) gained credibility in delivering national health information technology programs. Limitations of the program are its poor integration in the wider policy and quality improvement agenda of the Health Service Executive. The lack of specification of the sociotechnical elements of the program created challenges in communicating the program scope to key stakeholders and restricted the ability of program managers and implementers to test and refine the change package. This study concludes that while the sociotechnical elements of a national health information technology program do not need to be specified in tandem with technical elements, they do need to be specified early in the implementation process so that the change package used to implement the program can be tested and refined.

  • Source: StockSnap; Copyright: William Stitt; URL:; License: Public Domain (CC0).

    Uncovering a Role for Electronic Personal Health Records in Reducing Disparities in Sexually Transmitted Infection Rates Among Students at a Predominantly...


    Background: Black youth continue to bear an overwhelming proportion of the United States sexually transmitted infection (STI) burden, including HIV. Several studies on web-based and mobile health (mHealth) STI interventions have focused on characterizing strategies to improve HIV-related prevention and treatment interventions, risk communication, and stigma among men who have sex with men (MSM), people who use substances, and adolescent populations. The Electronic Sexual Health Information Notification and Education (eSHINE) Study was an exploratory mixed-methods study among students at a historically black university exploring perceptions on facilitating STI testing conversations with partners using electronic personal health records (PHRs). Objective: The purpose of this paper is to use eSHINE Study results to describe perceived impacts of electronic PHRs on facilitating STI testing discussions between sexual partners. Methods: Semistructured focus groups and individual in-depth interviews were conducted on a heterogeneous sample of students (n=35) between May and July 2014. Qualitative phase findings guided development of an online survey instrument for quantitative phase data collection. Online surveys were conducted using a convenience sample of students (n=354) between January and May 2015. Online survey items collected demographic information, sexual behaviors, beliefs and practices surrounding STI testing communication between partners, and beliefs about the impact of electronic PHR access on facilitating these discussions with partners. Chi-square analysis was performed to assess gender differences across quantitative measures. A Wilcoxon signed rank sum test was used to test the null hypothesis that electronic PHRs are believed to have no effect on the timing of dyadic STI health communication. Results: Participants described multiple individual and dyadic-level factors that inhibit initiating discussions about STI testing and test results with partners. Electronic PHRs were believed to improve ability to initiate conversations and confidence in STI screening information shared by partners. Among online survey participants, men were more likely to believe electronic PHRs make it easier to facilitate STI talks with potential partners (59.9% vs 51.9%; χ2=3.93, P=.05). The Wilcoxon signed-rank test results indicate significant increases in perceived discussion timing before sex with electronic PHR access (61.0% vs 40.4%; P<.001). Conclusions: Findings suggest that electronic PHR access in STI screening settings among similar populations of Black youth may improve both motivation and personal agency for initiating dyadic STI health communication. Results from this study will likely inform novel interventions that use access to electronic PHRs to stimulate important health-related discussions between sexual partners. Moving forward requires studying strategies for implementing interventions that leverage electronic PHRs to create new sexual health communication channels with providers, peers, and family among black youth.

  • Source: Freepik; Copyright: pressfoto; URL:; License: Licensed by JMIR.

    Task-Data Taxonomy for Health Data Visualizations: Web-Based Survey With Experts and Older Adults


    Background: Increasingly, eHealth involves health data visualizations to enable users to better understand their health situation. Selecting efficient and ergonomic visualizations requires knowledge about the task that the user wants to carry out and the type of data to be displayed. Taxonomies of abstract tasks and data types bundle this knowledge in a general manner. Task-data taxonomies exist for visualization tasks and data. They also exist for eHealth tasks. However, there is currently no joint task taxonomy available for health data visualizations incorporating the perspective of the prospective users. One of the most prominent prospective user groups of eHealth are older adults, but their perspective is rarely considered when constructing tasks lists. Objective: The aim of this study was to construct a task-data taxonomy for health data visualizations based on the opinion of older adults as prospective users of eHealth systems. eHealth experts served as a control group against the bias of lacking background knowledge. The resulting taxonomy would then be used as an orientation in system requirement analysis and empirical evaluation and to facilitate a common understanding and language in eHealth data visualization. Methods: Answers from 98 participants (51 older adults and 47 eHealth experts) given in an online survey were quantitatively analyzed, compared between groups, and synthesized into a task-data taxonomy for health data visualizations. Results: Consultation, diagnosis, mentoring, and monitoring were confirmed as relevant abstract tasks in eHealth. Experts and older adults disagreed on the importance of mentoring (χ24=14.1, P=.002) and monitoring (χ24=22.1, P<.001). The answers to the open questions validated the findings from the closed questions and added therapy, communication, cooperation, and quality management to the aforementioned tasks. Here, group differences in normalized code counts were identified for “monitoring” between the expert group (mean 0.18, SD 0.23) and the group of older adults (mean 0.08, SD 0.15; t96=2431, P=.02). Time-dependent data was most relevant across all eHealth tasks. Finally, visualization tasks and data types were assigned to eHealth tasks by both experimental groups. Conclusions: We empirically developed a task-data taxonomy for health data visualizations with prospective users. This provides a general framework for theoretical concession and for the prioritization of user-centered system design and evaluation. At the same time, the functionality dimension of the taxonomy for telemedicine—chosen as the basis for the construction of present taxonomy—was confirmed.

  • Source: Flickr; Copyright: US Department of Agriculture; URL:; License: Creative Commons Attribution (CC-BY).

    Adverse Drug Event Reporting From Clinical Care: Mixed-Methods Analysis for a Minimum Required Dataset


    Background: Patients commonly transition between health care settings, requiring care providers to transfer medication utilization information. Yet, information sharing about adverse drug events (ADEs) remains nonstandardized. Objective: The objective of our study was to describe a minimum required dataset for clinicians to document and communicate ADEs to support clinical decision making and improve patient safety. Methods: We used mixed-methods analysis to design a minimum required dataset for ADE documentation and communication. First, we completed a systematic review of the existing ADE reporting systems. After synthesizing reporting concepts and data fields, we conducted fieldwork to inform the design of a preliminary reporting form. We presented this information to clinician end-user groups to establish a recommended dataset. Finally, we pilot-tested and refined the dataset in a paper-based format. Results: We evaluated a total of 1782 unique data fields identified in our systematic review that describe the reporter, patient, ADE, and suspect and concomitant drugs. Of these, clinicians requested that 26 data fields be integrated into the dataset. Avoiding the need to report information already available electronically, reliance on prospective rather than retrospective causality assessments, and omitting fields deemed irrelevant to clinical care were key considerations. Conclusions: By attending to the information needs of clinicians, we developed a standardized dataset for adverse drug event reporting. This dataset can be used to support communication between care providers and integrated into electronic systems to improve patient safety. If anonymized, these standardized data may be used for enhanced pharmacovigilance and research activities.

  • Source: Wikimedia Commons; Copyright: Kgbo; URL:; License: Creative Commons Attribution + ShareAlike (CC-BY-SA).

    Nurses’ Experience With Health Information Technology: Longitudinal Qualitative Study


    Background: Nurses are the largest group of health information technology (HIT) users. As such, nurses’ adaptations are critical for HIT implementation success. However, longitudinal approaches to understanding nurses’ perceptions of HIT remain underexplored. Previous studies of nurses’ perceptions demonstrate that the progress and timing for acceptance of and adaptation to HIT varies. Objective: This study aimed to explore nurses’ experience regarding implementation of HIT over time. Methods: A phenomenological approach was used for this longitudinal qualitative study to explore nurses’ perceptions of HIT implementation over time, focusing on three time points (rounds) at 3, 9, and 18 months after implementation of electronic health records and bar code medication administration. The purposive sample was comprised of clinical nurses who worked on a medical-surgical unit in an academic center. Results: Major findings were categorized into 7 main themes with 54 subthemes. Nurses reported personal-level and organizational-level factors that facilitated HIT adaptation. We also generated network graphs to illustrate the occurrence of themes. Thematic interconnectivity differed due to nurses’ concerns and satisfaction at different time points. Equipment and workflow were the most frequent themes across all three rounds. Nurses were the most dissatisfied approximately 9 months after HIT implementation. Eighteen months after HIT implementation, nurses’ perceptions appeared more balanced. Conclusions: It is recommended that organizations invest in equipment (ie, wireless barcode scanners), refine policies to reflect nursing practice, and improve systems to focus on patient safety. Future research is necessary to confirm patterns of nurses’ adaptation to HIT in other samples.

  • Preterm infant in the NICU. Source: Flickr; Copyright: The Hudson Family; URL:; License: Creative Commons Attribution (CC-BY).

    The Impact of Implementation of a Clinically Integrated Problem-Based Neonatal Electronic Health Record on Documentation Metrics, Provider Satisfaction, and...


    Background: A goal of effective electronic health record provider documentation platforms is to provide an efficient, concise, and comprehensive notation system that will effectively reflect the clinical course, including the diagnoses, treatments, and interventions. Objective: The aim is to fully redesign and standardize the provider documentation process, seeking improvement in documentation based on ongoing All Patient Refined Diagnosis Related Group–based coding records, while maintaining noninferiority comparing provider satisfaction to our existing documentation process. We estimated the fiscal impact of improved documentation based on changes in expected hospital payments. Methods: Employing a multidisciplinary collaborative approach, we created an integrated clinical platform that captures data entry from the obstetrical suite, delivery room, neonatal intensive care unit (NICU) nursing and respiratory therapy staff. It provided the sole source for hospital provider documentation in the form of a history and physical exam, daily progress notes, and discharge summary. Health maintenance information, follow-up appointments, and running contemporaneous updated hospital course information have selected shared entry and common viewing by the NICU team. The interventions were to (1) improve provider awareness of appropriate documentation through a provider education handout and follow-up group discussion and (2) fully redesign and standardize the provider documentation process building from the native Epic-based software. The measures were (1) hospital coding department review of all NICU admissions and 3M All Patient Refined Diagnosis Related Group–based calculations of severity of illness, risk of mortality, and case mix index scores; (2) balancing measure: provider time utilization case study and survey; and (3) average expected hospital payment based on acuity-based clinical logic algorithm and payer mix. Results: We compared preintervention (October 2015-October 2016) to postintervention (November 2016-May 2017) time periods and saw: (1) significant improvement in All Patient Refined Diagnosis Related Group–derived severity of illness, risk of mortality, and case mix index (monthly average severity of illness scores increased by 11.1%, P=.008; monthly average risk of mortality scores increased by 13.5%, P=.007; and monthly average case mix index scores increased by 7.7%, P=.009); (2) time study showed increased time to complete history and physical and progress notes and decreased time to complete discharge summary (history and physical exam: time allocation increased by 47%, P=.05; progress note: time allocation increased by 91%, P<.001; discharge summary: time allocation decreased by 41%, P=.03); (3) survey of all providers: overall there was positive provider perception of the new documentation process based on a survey of the provider group; (4) significantly increased hospital average expected payments: comparing the preintervention and postintervention study periods, there was a US $14,020 per month per patient increase in average expected payment for hospital charges (P<.001). There was no difference in payer mix during this time period. Conclusions: A problem-based NICU documentation electronic health record more effectively improves documentation without dissatisfaction by the participating providers and improves hospital estimations of All Patient Refined Diagnosis Related Group–based revenue.

  • Source: Rawpixel; Copyright: Rawpixel; URL:; License: Public Domain (CC0).

    Validation of a Natural Language Processing Algorithm for Detecting Infectious Disease Symptoms in Primary Care Electronic Medical Records in Singapore


    Background: Free-text clinical records provide a source of information that complements traditional disease surveillance. To electronically harness these records, they need to be transformed into codified fields by natural language processing algorithms. Objective: The aim of this study was to develop, train, and validate Clinical History Extractor for Syndromic Surveillance (CHESS), an natural language processing algorithm to extract clinical information from free-text primary care records. Methods: CHESS is a keyword-based natural language processing algorithm to extract 48 signs and symptoms suggesting respiratory infections, gastrointestinal infections, constitutional, as well as other signs and symptoms potentially associated with infectious diseases. The algorithm also captured the assertion status (affirmed, negated, or suspected) and symptom duration. Electronic medical records from the National Healthcare Group Polyclinics, a major public sector primary care provider in Singapore, were randomly extracted and manually reviewed by 2 human reviewers, with a third reviewer as the adjudicator. The algorithm was evaluated based on 1680 notes against the human-coded result as the reference standard, with half of the data used for training and the other half for validation. Results: The symptoms most commonly present within the 1680 clinical records at the episode level were those typically present in respiratory infections such as cough (744/7703, 9.66%), sore throat (591/7703, 7.67%), rhinorrhea (552/7703, 7.17%), and fever (928/7703, 12.04%). At the episode level, CHESS had an overall performance of 96.7% precision and 97.6% recall on the training dataset and 96.0% precision and 93.1% recall on the validation dataset. Symptoms suggesting respiratory and gastrointestinal infections were all detected with more than 90% precision and recall. CHESS correctly assigned the assertion status in 97.3%, 97.9%, and 89.8% of affirmed, negated, and suspected signs and symptoms, respectively (97.6% overall accuracy). Symptom episode duration was correctly identified in 81.2% of records with known duration status. Conclusions: We have developed an natural language processing algorithm dubbed CHESS that achieves good performance in extracting signs and symptoms from primary care free-text clinical records. In addition to the presence of symptoms, our algorithm can also accurately distinguish affirmed, negated, and suspected assertion statuses and extract symptom durations.

  • Source: Freepik; Copyright: peoplecreations; URL:; License: Licensed by JMIR.

    Health Information Technology in Healthcare Quality and Patient Safety: Literature Review


    Background: The area of healthcare quality and patient safety is starting to use health information technology to prevent reportable events, identify them before they become issues, and act on events that are thought to be unavoidable. As healthcare organizations begin to explore the use of health information technology in this realm, it is often unclear where fiscal and human efforts should be focused. Objective: The purpose of this study was to provide a foundation for understanding where to focus health information technology fiscal and human resources as well as expectations for the use of health information technology in healthcare quality and patient safety. Methods: A literature review was conducted to identify peer-reviewed publications reporting on the actual use of health information technology in healthcare quality and patient safety. Inductive thematic analysis with open coding was used to categorize a total of 41 studies. Three pre-set categories were used: prevention, identification, and action. Three additional categories were formed through coding: challenges, outcomes, and location. Results: This study identifies five main categories across seven study settings. A majority of the studies used health IT for identification and prevention of healthcare quality and patient safety issues. In this realm, alerts, clinical decision support, and customized health IT solutions were most often implemented. Implementation, interface design, and culture were most often noted as challenges. Conclusions: This study provides valuable information as organizations determine where they stand to get the most “bang for their buck” relative to health IT for quality and patient safety. Knowing what implementations are being effectivity used by other organizations helps with fiscal and human resource planning as well as managing expectations relative to cost, scope, and outcomes. The findings from this scan of the literature suggest that having organizational champion leaders that can shepherd implementation, impact culture, and bridge knowledge with developers would be a valuable resource allocation to consider.

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Latest Submissions Open for Peer-Review:

View All Open Peer Review Articles
  • Classification of excluded or missing virology data from UK-wide influenza vaccine effectiveness studies

    Date Submitted: Sep 11, 2018

    Open Peer Review Period: Sep 16, 2018 - Nov 11, 2018

    Background: Monitoring the effectiveness of the influenza vaccination programme within the UK is necessary in order to assess its clinical impact. Data are collected from general practice sentinel net...

    Background: Monitoring the effectiveness of the influenza vaccination programme within the UK is necessary in order to assess its clinical impact. Data are collected from general practice sentinel network computerised medical record (CMR) systems on patients from whom virology specimens have been taken for influenza. The data collected includes demographics, comorbidities, vaccine exposure and if patients have had a virology specimen taken. Unfortunately not all virology specimens collected can be used in the vaccine effectiveness (VE) studies conducted. Objective: To describe the proportion, reasons and any trends in virology specimen data collected but not used in influenza VE analyses, with the goal of defining strategies to reduce collection of specimens ineligible for use in VE studies. Methods: We examined UK influenza VE studies from the past 10 years and identified incidences where data were labelled unusable. We categorised reasons for not using data as: (1) Vaccination history: Missing or Uncertain categories (including patient not registered with the practice at the start of the season); (2) Swab timing: Not recorded; More than 7 days (historically over 29 days) after symptom onset or within 14 days of vaccination; (3) Laboratory: Not sufficient data for processing (e.g. no age), CT values; (4) Flu or vaccination type of no interest (including pandemic years). The proportion, reasons and trends for data loss were identified through descriptive statistics and graphical representations. We included an analysis of where other data had been available at the point of analysis but not used. Results: Over 30% (13292/41337) of virology specimen data was not used across all seasons. Data loss gradually began to decrease from 2014/15 onwards. Data loss were highest for flu or vaccination type of no interest and swab timing. Retrospective and prospective actions were identified to reduce data loss in future. Around 60% of samples could have been included if identifiable data were better shared between records. Conclusions: The reasons for excluding samples and missing data varied, particularly prior to 2014; consistent categorisation was in place from 2014 onwards. Leaving aside the different issues around pandemic years, many of the virology swabs not included were due to suboptimal case selection by practices, but over half (58%) could have been included if identifiable data were better shared between data sources. Clinical Trial: N/A

  • Anesthesiology Control Tower: Feasibility Assessment to Support Translation (ACTFAST) a mixed methods study of a novel telemedicine-based support system for the operating room

    Date Submitted: Sep 9, 2018

    Open Peer Review Period: Sep 14, 2018 - Nov 9, 2018

    Background: Major postoperative morbidity and mortality remain common despite efforts to improve patient outcomes. Health information technologies, such as decision support systems, have the potential...

    Background: Major postoperative morbidity and mortality remain common despite efforts to improve patient outcomes. Health information technologies, such as decision support systems, have the potential to advance the standard of perioperative patient care. Failure to evaluate the usability of these technologies and barriers to their implementation can limit their acceptance within health systems. Objective: This manuscript describes the usability and acceptability of and systematic process for developing and adapting an innovative telemedicine based clinical support system, the Anesthesiology Control Tower. It also reports stakeholders’ perceptions of the barriers and facilitators the implementation of the intervention. Methods: Three phases of testing were conducted in an iterative manner in order to evaluate both the individual components of the Anesthesiology Control Tower and their integration as a whole. Phase 1 testing employed a “think-aloud” protocol analysis to identify surface level usability problems with individual software components of the ACT, in addition to the entirety of the structure. Phase 2 testing involved an extended qualitative and quantitative in-situ usability analysis. Phase 3 sought to identify major barriers and facilitators to implementation of the ACT through semi-structured interviews with key stakeholders. Results: Numerous usability problems with the software components of the ACT were identified in the Phase 1 and Phase 2 usability testing sessions. In response to these problems, seven iterations of the ACT software platform were developed. Initial satisfaction with the ACT, as measured by standardized measures, was below commonly accepted cutoffs for these measures. Satisfaction improved to acceptable levels over the course of revision and testing. A number of barriers to implementation were identified and addressed during the refinement of the ACT intervention. Conclusions: The Anesthesiology Control Tower system has the potential to improve the standard of perioperative anesthesia care. Through our thorough and iterative usability testing process and stakeholder assessment of barriers and facilitators, we were able to maximize the acceptability of this novel technology, thus improving our ability to implement this innovation into the model of care for perioperative medicine.

  • Lessons Learned from Implementing a Heart Failure Telemonitoring System in Home Care Nursing

    Date Submitted: Aug 21, 2018

    Open Peer Review Period: Aug 27, 2018 - Oct 22, 2018

    Background: Telemonitoring (TM) of heart failure patients in a clinic setting has been shown to be effective if properly implemented, but little is known about the feasibility and impact of implementi...

    Background: Telemonitoring (TM) of heart failure patients in a clinic setting has been shown to be effective if properly implemented, but little is known about the feasibility and impact of implementing TM through a home care nursing agency. Objective: The goal of this study was to determine the feasibility of implementing a smartphone-based TM system through a home care nursing agency, and to explore the feasibility of conducting a future effectiveness trial. Methods: A feasibility study was conducted, aiming to recruit 10-15 heart failure patients who would use the TM system for 4 months by taking daily measurements of weight and blood pressure, and recording symptoms. Home care nurses responded to alerts generated by the TM system either through a phone call and/or home visit. Results: Only six patients were recruited over a six-month period due to lack of referrals from physicians. Potential benefits of TM through a home care nursing agency were indicated, including through improved patient education, providing nurses with a better understanding of the patient’s health status, and reductions in home visits. Barriers to implementation included challenges in nurses contacting patients and physicians, retention issues, and integrating the TM system into a complex home care nursing workflow. Conclusions: Lessons learned included the need to incentivize physicians, to ensure streamlined processes for recruitment and communication, to target appropriate patient populations, and to create a core clinical group. Barriers encountered in this feasibility trial should be considered to determine their applicability when deploying innovations into different service delivery models.

  • Opportunities and Challenges of Telehealth in Remote Communities: A Case Study of the Yukon Telehealth System

    Date Submitted: Aug 20, 2018

    Open Peer Review Period: Aug 25, 2018 - Oct 20, 2018

    Background: Telehealth has been shown to improve access to healthcare and to reduce costs to the patient and healthcare system, especially for patients living in rural settings. However, unique challe...

    Background: Telehealth has been shown to improve access to healthcare and to reduce costs to the patient and healthcare system, especially for patients living in rural settings. However, unique challenges arise when implementing telehealth in remote communities. Objective: The objectives of this evaluation were to understand the current use, challenges, and opportunities of the Yukon Telehealth System. The lessons learned from this case study were used to determine important factors to consider when attempting to advance and expand telehealth programs in remote communities. Methods: A mixed-methods approach was used to evaluate the Yukon Telehealth System and to determine possible future advances. Quantitative data were obtained through usage logs. In addition, online questionnaires were administered to nurses in each of the 14 Yukon community health centres outside of Whitehorse, and patients who had used telehealth were also asked to complete a questionnaire. Qualitative data included focus groups and semi-structured interviews with a total of 36 telehealth stakeholders. Results: Since 2008, there have been a consistent total number of telehealth sessions of about 1000 per year, with the main use being for clinical care (70% of all sessions in 2015). From the questionnaire data (11 community nurses, 10 patients) and interview data, there was a consensus among the clinicians and patients that the System provided timely access and cost savings from reduced travel. However, they believed that it was underutilized and the equipment was outdated. The challenges and opportunities discovered led to the identification of four factors that should be considered when trying to advance and expand a telehealth program. 1) Patient and clinician buy-in: Past telehealth experiences should be considered when advancing the system, such as negative clinician experiences with outdated technology. Expansion of services in orthopaedics, dermatology, and psychiatry were found to have particular benefit in Yukon by clinicians specializing in these areas. 2) Workflow: The use and scheduling of telehealth should be streamlined and automated as much as possible to reduce dependencies on the single Yukon Telehealth Coordinator. 3) Access to telehealth technology: Clinicians and patients should have easy access to telehealth technology, whether it is telehealth units or alternative desktop applications. The use of consumer products, such as mobile technology, should be leveraged as appropriate. 4) Infrastructure: The required human resources and technology need to be established when expanding and advancing telehealth. Conclusions: While clinicians and patients have generally positive perceptions of the Yukon Telehealth System, there was consensus that it was underutilized. Many opportunities exist to significantly expand the types of telehealth services and the number of telehealth sessions. The lessons learned from this evaluation can be applied to other remote communities to realize telehealth’s potential as a means for efficient, safe, convenient, and cost-effective care delivery.

  • A new method for estimating morbidity rates based on routine electronic medical records in primary care

    Date Submitted: Aug 13, 2018

    Open Peer Review Period: Aug 19, 2018 - Oct 15, 2018

    Background: Routinely recorded electronic health records (EHRs) from general practitioners (GPs) are increasingly available and provide valuable data for estimating incidence and prevalence rates of d...

    Background: Routinely recorded electronic health records (EHRs) from general practitioners (GPs) are increasingly available and provide valuable data for estimating incidence and prevalence rates of diseases in the population. Objective: This paper describes how we developed an algorithm to construct episodes of illness based on EHR data to calculate morbidity rates. Methods: The algorithm was developed in discussion rounds with two expert groups and tested with data from NIVEL Primary Care Database, which consisted of a representative sample of 219 general practices, covering a total population of 867,140 listed patients in 2012. Morbidity data were used from EHRs in the period 2010-2012, including recorded ICPC coded episodes of care, encounters and prescriptions. Results: All 685 symptoms and diseases of ICPC-1 were categorized as acute symptoms/diseases, long-lasting reversible diseases, and chronic diseases. Based on knowledge of the duration of a disease, for each category an algorithm was developed to construct episodes of illness. Compared with recorded episodes of care, for acute and long-lasting diseases, applying the algorithm resulted in a reduction of both the number and average duration of the episodes up to 53% and 94%, respectively. On the other hand, for chronic diseases, the algorithm resulted in a slight increase in the number of episodes as well as the episode duration. Conclusions: An algorithm was developed to construct episodes of illness based on routinely recorded EHR data to estimate morbidity rates. The algorithm constitutes a simple and uniform way of using EHR data and can easily be applied in other registries.

  • Patient's Satisfaction with Dental Service Provided by Dental Students of Qassim University, KSA.

    Date Submitted: Aug 13, 2018

    Open Peer Review Period: Aug 15, 2018 - Oct 11, 2018

    Background: In the past, dental patient satisfaction has been measured by the dentist's technical competence and mechanical precision2. Patient's opinion played no part in this method of measuring qua...

    Background: In the past, dental patient satisfaction has been measured by the dentist's technical competence and mechanical precision2. Patient's opinion played no part in this method of measuring quality. Consequently, consideration of patient's satisfaction became an integral part of providing dental services3. Some patients want a dentist who listens to them, has a friendly caring attitude, discuses treatment options and procedures, and gives confidence7. Communication skills, a component of empathy, have been shown to be important in limiting patient's dissatisfaction12. The self-rated oral health status might be considered as the key factor of patient’s satisfaction and positive feedback. Objective: To achieve high levels of patient's satisfaction, it is crucial to find out which items patients consider important and how they operationalize these items 17. Finally, medical care cannot be of high quality, unless patients’ satisfaction is observed. Methods: A 22-item patient's satisfaction questionnaire was designed. The questionnaire was used to obtain data from patients who were then currently receiving care in the College of Dentistry, Qassim University, KSA after minimum of three visits to the dental chair. Two hundred and fifty copies of the questionnaire were distributed equally in both male and female sections over four months period (March-August 2017). Results: A total of 250 questionnaire papers were distributed of which 215 were returned for a response rate of 86%. The response rate in male section was (94%) higher than that in female section (78%) at College of Dentistry, Qassim University, KSA. Overall ranking of factors related to satisfaction was done by calculating the mean percentage of agreement regarding the different disciplines of satisfaction among the studied group. The mean percentage of agreement for the 3 disciplines was 82.5% denoting a high level of satisfaction. Conclusions: Results indicated that the majority of patients were satisfied with the provided dental service at College of Dentistry, Qassim University, KSA. Patient dentist interaction and Technical competency had the most significant effect on patient satisfaction levels in this study . On the other hand, Administrative efficiency (AE) and Clinical setup environment (CS) were the factor that had the most impact on levels of dissatisfaction. In this study, no significant differences were observed between the satisfaction score and Nationality of the patients. The only significant finding observed for the satisfaction score was gender .While there was a significant differences regarding satisfaction between both genders . Clinical Trial: The Research Ethics Committee and Research & Presentation Skills Supervision Committee at College of Dentistry, Qassim University, KSA approved this study (Code #:EA/203/2017) and (SRPSSC #: M-5003-17) . The waiver of the informed consent process was approved on the basis of the questionnaires being anonymous and self-administered and containing no identifiers.