This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.
Surveillance of health care-associated infections is an essential component of infection prevention programs, but conventional systems are labor intensive and performance dependent.
To develop an automatic surveillance and classification system for health care-associated bloodstream infection (HABSI), and to evaluate its performance by comparing it with a conventional infection control personnel (ICP)-based surveillance system.
We developed a Web-based system that was integrated into the medical information system of a 2200-bed teaching hospital in Taiwan. The system automatically detects and classifies HABSIs.
In this study, the number of computer-detected HABSIs correlated closely with the number of HABSIs detected by ICP by department (n=20;
This system provides reliable and objective HABSI data for quality indicators, improving the delay caused by a conventional surveillance system.
Health care-associated infections (HAIs), adverse events related to health care, excess mortality and morbidity, and resource use are responsible for augmenting antimicrobial resistance [
Attention to HAIs has increased partially because of legislative mandates for reporting and reimbursement policies [
In continuation with our previous efforts in developing a Web-based MDRO surveillance system that automatically identifies and accurately detects suspicious outbreaks of MDROs [
The study was conducted at NTUH, a 2200-bed teaching hospital that provides primary and tertiary care for the adult and the pediatric population in Taiwan. This study was approved by the Institutional Ethics Review Board of NTUH (NTUH-200904014R). In 2011, NTUH served 2,309,108 outpatients, received 106,950 emergency visits, and discharged 104,899 patients (723,505 patient-days).
Two sets of blood samples from separate venipuncture sites for bacterial culture were routinely collected from patients who were newly diagnosed with sepsis. An additional sample was collected after 45-60 minutes to define continuous bloodstream infection (BSI). Only 1 blood sample for a follow-up culture was collected to confirm the clearance of BSI. Of the 80,327 blood specimens that were sent for isolation and identification of pathogens, 991 (1.23%) were obtained through a single blood draw. A total of 8745 samples grew 1 or more pathogens (10.88%); of these, 1908 exhibited HABSIs. The pooled mean of HABSI incidence was 14.7 episodes/1000 patients (range 0.2-112.7/1000 patients by department) and 2.13 episodes/1000 patient-days.
A prospective, hospital-wide on-site surveillance of HAIs, initiated in 1981 [
The key data are collected by systematically reviewing hospital information systems (HISs), laboratory information systems (LISs), and handwritten charts. Because of advances in medical information system, data of HISs and LISs are currently stored as electronic medical records (EMRs). However, data generated in the medical information system are scattered in numerous databases, and data access is hindered by several interfaces. In addition, data must be integrated, interpreted, and transformed into meaningful information.
We established a rule-based HABSI surveillance and classification system (the system), which was implemented on October 1, 2010. The current version was revised on September 20, 2012 (
This data-collection subsystem collects relevant data from HIS and LIS using HL7 standards, which was extended stepwise from the previous version [
To develop the detection rules in the candidate-detection subsystem, the ICPs have adapted objective components of the National Health Care Safety Network (NHSN) definitions established by the CDC [
The HAI management subsystem, established on July 1, 2007 [
General architecture of the Web-based health care-associated infection (HAI) surveillance and classification system. DB: database; HL7: Health Level Seven; SOA: service-oriented architecture.
Computer algorithms to detect health care-associated bloodstream infection (HABSI) by active daily screening of data from hospital information system and laboratory information system. HABSIs are classified into primary HABSI (PRIM), secondary HABSI (SEC), and clinical sepsis (CSEP) as described in
Clinically useful tests must be valid and reliable and have a reasonable turnaround time. Thus, we conducted a 3-aspect evaluation, including accuracy, reliability, and efficiency, of the system.
We first evaluated the performance of the system during the developmental phase (ie, before implementation of the system) regarding its potential to provide data for quality indicators. Computer-detected HABSIs were compared with ICP-detected HABSIs as a routine practice between July 1, 2010, and September 30, 2010. The correlation between these 2 data sources was analyzed according to department distribution and time trend of HABSIs.
On the basis of inconsistent and varied performances of the conventional ICP-based surveillance system, we further evaluated the performance after implementing the system using ICP-defined reference standards. To generate high-quality reference standards, 11 ICPs performed a retrospective review of all medical data of patients who were admitted between October 1, 2012, and October 31, 2012, to identify HABSI cases based on NTUH detection rules, and one of the authors (H-CL) validated the results. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) of the system, and Cohen kappa coefficient [
We then compared the delay in HABSI confirmation (as a proxy for practice efficiency) before (October 2007-September 2010) and after (October 2010-September 2013) the system implementation. The delay in confirmation was defined as the intervals between the HABSI confirmation dates (complete data entry and confirmed by ICPs in the HAI management subsystem) and log-in dates of the first blood specimen with positive results and was calculated by averaging the number of delay days in each month. The study periods were selected taking into account the seasonal variation of HAI rates demonstrated previously [
Timeline of development and performance evaluation of the health care-associated bloodstream infections (HABSIs) surveillance and classification system. ICP: infection control personnel.
During the 14-week study period (July 1, 2010-September 30, 2010), 501 episodes of ICP-detected HABSIs and 479 episodes of computer-detected HABSIs were found throughout the 20 departments. The data were highly correlated by place and time (
Correlation of 501 episodes of infection control personnel (ICP)-detected health care-associated bloodstream infection (HABSI) and 479 episodes of computer-detected HABSI from 20 departments during the 14-week study period. (A) Perfect agreement of HABSI episodes by department (n=20, Pearson correlation,
In October 2012, the system identified 167 episodes of HABSIs (
Comparison of the case detection results of the health care-associated bloodstream infection surveillance and classification system with infection control personnel reference standard between the periods October 1 and October 31, 2012.
Infection control personnel reference standard | Automated surveillance classification |
|
|
HABSI | Not HABSI | Total | |
HABSI | 160 | 3a | 163 |
Not HABSI | 7b | 17,824 | 17,831 |
Total | 167 | 17,827 | 17,994 |
aRetrospective review by 2 investigators independently confirmed that these 3 episodes of HABSI due to common skin commensals were missed due to fever criteria (temperature > 38°C): 1 patient received antipyretic agents, 1 with a and sustained increase in temperature (>1°C) from baseline but less than 38°C, and in the other patient fever was documented only in the progress note and was missed by using this fever criteria.
bFour false-positive cases due to revision of final laboratory reports after “recall day.” One episode of community-acquired BSI was detected as HABSI due to delay in transportation of specimen to microbiology laboratory. Two were cases of persistent bloodstream infection.
Computer algorithms identifying 167 events of health care-associated bloodstream infection among 8862 inpatients and 9132 patients in the emergency department between October 1 and October 31, 2012 (31 days).
The delay in HABSI confirmation was reduced from 43.58 (SD 15.57) days before the system implementation (October 2007-September 2010, 1096 days) to 14.58 (SD 4.64) days after the implementation (October 2010-September 2013, 1096 days;
The detection delay of health care-associated bloodstream infection (HABSI) decreased gradually from July 2007 to December 2013. The first version of health care-associated infection (HAI) management subsystem has been developed to facilitate infection control personnel-based surveillance program since July 2007. This was revised stepwise and has been operation as an automatic system since October, 2010. In March 2009, this hospital initiated preparedness for international hospital accreditation, which was scheduled 1 year later. Influenza pandemic occurred in April 2009.
This Web-based, automated HABSI surveillance and classification system used discrete data elements obtained from HISs, and LISs provided data highly correlated with conventional ICP surveillance system. The performance was excellent regarding sensitivity, specificity, PPV, and NPV, and was in agreement with reference standards; the system reduced the delay in confirmation, on average, by 29 days. The system improves practice efficiency, enabling ICPs to intensify intervention and further reduce HAI rates.
Computer-assisted HAI surveillance and classification systems are widely implemented [
In this study, the HABSI detection rules (see
The system detects and classifies HABSIs automatically and ensures professional autonomy by requiring further confirmation. Each episode of HAI requires confirmation by ICPs. The system presents detailed information about each HABSI candidate systematically to support decision making. The main reason for this design is because the system is imperfect (see the “Limitations” section). Furthermore, because HAIs are rare in hospitalized patients, the system aims to select potential HAI candidates and exclude patients who do not have an HAI and hence do not require review by ICPs.
This study verifies the potential of the system to provide data for quality indicators. The system enabled sustainable surveillance, generating data that were correlated with conventional surveillance systems by department and time. In addition, the delay in HABSI confirmation decreased to 5.78 (SD 0.91) days in the second half of 2013. Because of the reduced length of hospital stay and the increased threat of emerging infectious diseases, early detection of HAIs can enable identifying the reservoir or index case and providing early intervention before pathogens spread further. Currently, the delay in HABSI confirmation is caused by the time required to identify the positive blood cultures and microorganisms; the system detects and analyzes results of blood cultures to prevent false alerts. Furthermore, the major challenge encountered when sharing automated HABSI surveillance systems between hospitals is different HIS settings [
Although our results suggested that the system performs well, this study had several limitations. First, data integrity and instantaneity substantially affect performance, because this system uses EMRs from many sources. In addition, not all data required for HAI surveillance [
Fourth, the agreement regarding the place of onset (responsible ward) was not evaluated in this study, because patients are frequently transferred to different wards and electronic clinical data regarding symptoms and signs of infection were unavailable during the study period. Fifth, the case definition of HABSI is complicated when clinical scenarios are taken into consideration. For disagreements and received revaluations (
This fully automated system that can be integrated in medical information systems detects and classifies HABSI within 5.78 (SD 0.91) days after occurrence, enabling the opportunity for early intervention. Currently, the system and other components of the infection control system [
Detection rules for the health care-associated bloodstream infection (HABSI) Surveillance and Classification System at National Taiwan University Hospital (NTUH) and corresponding Centers for Disease Control and Prevention (CDC) National Healthcare Safety Network (NHSN) definition of health care-associated infection (HAI) [
The list of central lines and intravascular devices.
The definition of symptoms/signs of sepsis in pediatric patients.
Summary of characteristics and performances of computer-assisted health care-associated infection surveillance systems in the literature.
User interface of HAI Management System. This system integrated all the information which was needed for HAI decision.
central line-associated bloodstream infection
Disease Control and Prevention
central line-related bloodstream infection
electronic medical records
health care-associated bloodstream infection
health care-associated infections
hospital information systems
Health Level Seven
infection control personnel
laboratory information systems
multidrug-resistant organism
National Health Care Safety Network
negative predictive value
National Taiwan University Hospital
positive predictive value
service-oriented architecture
The authors are grateful to the members of the Center of Infection Control and the Information Systems Office at National Taiwan University Hospital for their contributions to this study, and to Kenneth D Mandl for his critical review and comments. This work was supported by grants from the Department of Health, Taiwan (Grant Nos DOH98-DC-1007 and DOH100-TD-B-111-001 to Y-CC) and the National Science Council, Taiwan (Grant Nos NSC 101-2221-E-002 -203 -MY3 to FL and NSC 103-2917-I-564-063 to Y-JT). The funding organizations were not involved in designing or conducting the study, data collection, management, analysis, or interpretation, or in the preparation, review, and approval of the manuscript.
None declared.