This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.
Pressure ulcers (PUs) are considered a serious problem in nursing care and require preventive measures. Many risk assessment methods are currently being used, but most require the collection of data not available on admission. Although nurses assess the Nursing Needs Score (NNS) on a daily basis in Japanese acute care hospitals, these data are primarily used to standardize the cost of nursing care in the public insurance system for appropriate nurse staffing, and have never been used for PU risk assessment.
The objective of this study was to predict the risk of PU development using only data available on admission, including the on-admission NNS score.
Logistic regression was used to generate a prediction model for the risk of developing PUs after admission. A random undersampling procedure was used to overcome the problem of imbalanced data.
A combination of gender, age, surgical duration, and on-admission total NNS score (NNS group B; NNS-B) was the best predictor with an average sensitivity, specificity, and area under receiver operating characteristic curve (AUC) of 69.2% (6920/100), 82.8% (8280/100), and 84.0% (8400/100), respectively. The model with the median AUC achieved 80% (4/5) sensitivity, 81.3% (669/823) specificity, and 84.3% AUC.
We developed a model for predicting PU development using gender, age, surgical duration, and on-admission total NNS-B score. These results can be used to improve the efficiency of nurses and reduce the number of PU cases by identifying patients who require further examination.
The National Pressure Ulcer Advisory Panel (NPUAP)/
PUs result in excessive hospital lengths of stay [
The Japanese Society of Pressure Ulcers considers shock, surgical duration of more than six hours, and peripheral circulatory insufficiency among the factors that contribute to a high risk of developing PUs [
Nurses usually use a risk assessment scale, such as the Waterlow or Braden scale, to identify high-risk patients, reviewed in [
Tokushima University Hospital established a team of PU specialists in 2007 to detect early PU cases and prevent their advancement. An interdisciplinary team composed of plastic and reconstructive surgeons; wound, ostomy, and continence nurses; a medical informatics engineer; a physical therapist; and others have been designated for this purpose. They investigate data of PU patients and discuss countermeasures on a weekly basis. All inpatients in Tokushima University Hospital are assessed for their PU risk using a nonstandard procedure requiring the collection of additional items. High-risk inpatients are followed up according to our hospital PU risk assessment protocol with the Braden scale. A method that can easily and accurately identify high-risk patients for further inspection using only data available on admission would be highly beneficial.
NNS items in general wards.
NNS-Aa | Monitoring and treatment | Score | NNS-Bb | Patient condition | Score | ||||
0 | 1 | 2 | 0 | 1 | 2 | ||||
1 | Wound treatment | No | Yes |
|
10 | Turn over | Able | Partially able | Unable |
2 | Blood pressure measurement | 0-4 times | More than 5 times |
|
11 | Sit up | Able | Unable |
|
3 | Urine volume measurement | No | Yes |
|
12 | Keep a sitting position | Able | Partially able | Unable |
4 | Respiratory care | No | Yes |
|
13 | Transfer activity | Able | Partially able | Unable |
5 | ≥3 Intravenous lines | No | Yes |
|
14 | Oral care | Able | Unable |
|
6 | Electrocardiogram monitor | No | Yes |
|
15 | Feed self | No assistance | Partial assistance | Full assistance |
7 | Syringe pump | No | Yes |
|
16 | Change clothes | No assistance | Partial assistance | Full assistance |
8 | Blood transfusion or blood derivative | No | Yes |
|
|
|
|
|
|
9 | Specialized treatmentc | No |
|
Yes |
|
|
|
|
|
a NNS-A=Nursing Needs Score-A
b NNS-B=Nursing Needs Score-B
c 1, antineoplastic agent; 2, narcotic injection; 3, radiation therapy; 4, immunosuppressive agent; 5, vasopressor agent; 6, antiarrhythmic agent; and 7, drainage management
Many methods have been implemented for standardizing medical cost calculations to comply with the Japanese medical insurance system that includes all residents in Japan. In acute care hospitals, one method used to evaluate nursing costs is the NNS that was introduced first to intensive care units in 2003, then to high-care units in 2004, and finally to general wards in 2008 (
Nurses are the principal specialists for risk assessment of in-hospital acquired PU. We aimed to develop a quick and simple PU prediction tool that uses only data documented by nurses on patient admission to estimate the risk of PU development. Patients identified as being at high risk can be further investigated. With this tool, there would be no need for nurses to assess factors such as a patient’s moisture, friction, or shear state using the Braden scale, or skin type, weight loss, or continence using the Waterlow scale. Rather, early predictions can be made using the data available on admission with regard to the patient’s risk of developing PUs. Those identified as being at high risk could then be identified and followed more closely.
The institutional review board of Tokushima University approved the study protocol, and opt-out consent was obtained. This was a retrospective study with respect to PU prediction for all inpatients that had their NNS recorded on admission in Tokushima University Hospital (696 beds), an acute care hospital, from January 1 to December 31, 2012. This study assessed data pertaining to demographic characteristics, surgical duration, and NNS data collected from the HIS, and PU data recorded by PU specialists. The total number of PUs per patient and PU stages were categorized from “depth unknown” to Stage IV (full thickness tissue loss), as defined by NPUAP staging guidelines [
The data we collected suffer, as do many other medical data, from class imbalance. In an imbalanced dataset, the number of one class is much higher than the other. This occurs primarily because of a high prior probability of one class and a low prior probability of the other class. The dataset used has 8235 patients without in-hospital PU and 51 cases with in-hospital PU, resulting in a high imbalance with a positive to negative ratio of 1 to 162.
Descriptive statistics of patient age.
Gender | PU-positive | PU-negative | |||
|
|
n | Age |
n | Age |
Male | 35 | 62.1 (14.1) | 4566 | 62.6 (15.6) | |
Female | 16 | 67.2 (14.7) | 3669 | 59.4 (18.0) |
Descriptive statistics of patient surgical duration.
Surgery | PU-positive | PU-negative | |||
|
n | Duration (hours) |
n | Duration (hours) |
|
Yes | 28 | 6.5 (4.4) | 3158 | 2.7 (2.2) | |
No | 23 | - | 5077 | - |
Descriptive statistics of patient NNS.
PU-positive | PU-negative |
NNS-A |
NNS-B |
0.5 (1.0) | 0.3 (0.8) |
3.2 (4.0) | 1.1 (2.3) |
Distribution of total Nursing Needs Score group A =NNS-A, and Nursing Needs Score group B = NNS-B scores for pressure ulcers patients.
Logistic regression analysis is commonly used to determine the relationship between different qualitative and quantitative independent variables and a qualitative dependent variable. In this study, the dependent variable was whether the patient developed an in-hospital PU or not. The logistic regression model was generated using the Weka logistic regression component in RapidMiner version 5.3, Community version (RapidMiner, Inc, USA).
Standard data analysis procedures do not apply to imbalanced datasets [
Data were randomly divided into two sets. The first set (training dataset) includes 90.00% (7412/8235 patients without in-hospital PU) and 90% (46/51 cases with in-hospital PU) of each class and was used for model generation, and the second set (test dataset) consists of the remaining 9.99% (823/8235 patients without in-hospital PU) and 9% (5/51 cases with in-hospital PU) of each class and was used to evaluate the generated model (
Using the above-described method, the following three sets of factors were examined to determine the most predictive dataset.
The ΣA set of factors were gender, age, surgical duration, and total NNS-A score
The ΣB set of factors were gender, age, surgical duration, and total NNS-B score
The ΣAB set of factors were gender, age, surgical duration, total NNS-A score, and total NNS-B score.
Model generation and evaluation process.
Our aim was to use the generated logistic regression model for classification, so the classification table is the most appropriate evaluation method [
It is important to have a model with both high sensitivity and high specificity, since a low sensitivity means the model will not efficiently predict the more important positive class, while having a low specificity means the model will have many false positives. A measurement that combines both sensitivity and specificity is the area under receiver operating characteristic curve (AUC). A high AUC indicates the model has both high sensitivity and specificity. In this study, we used AUC to identify models with high sensitivity and specificity.
Sensitivity and specificity.
Mean performance of different factor sets.
Factor set | Accuracy mean (SD) | Sensitivity mean (SD) | Specificity mean (SD) | AUC mean (SD) |
ΣA | 82.2 (2.8) | 47.6 (10.5) | 82.4 (2.8) | 71.4 (4.5) |
ΣB | 82.7 (2.0) | 69.2 (11.5) | 82.8 (2.1) | 84.0 (3.3) |
ΣAB | 82.1 (2.3) | 70.6 (10.8) | 82.2 (2.3) | 84.3 (3.9) |
The model that provided the closest results to average was the model with the median AUC (3) (
Logistic regression model of the median AUC.
It is worth mentioning that only four of the 100 models gave minimum sensitivity using the ΣB factor set, which indicates that the random undersampling procedure was effective in overcoming class imbalance and will provide a good prediction model in most cases.
We examined the possibility of predicting PU development from data typically recorded by nurses on patient admission, such as gender, age, surgical duration, and total NNS scores. Being able to achieve this would make efficient use of the existing HIS system and be of great benefit to nurses. An argument can be made that using other hospitalization data (eg, skin and support surface status, special mattresses, laboratory test results, malnutrition, etc) might improve prediction accuracy; however, our aim is to provide a system that can predict PU risk on admission without the need to collect additional data. An experienced nurse can observe changes in a patient’s status and provide an effective assessment, while a novice nurse may not be able to. With PU assessment on admission, a novice nurse can identify patients likely to develop PUs during hospitalization. Studies have shown that education [
Follow-up procedures will be needed for patients identified as being at high risk of developing PUs based on admission data. Our hospital uses the Braden scale to follow high-risk patients, which includes assessment subscales for mobility/activity risk factors [
Notably, gender requires further consideration. We found that, consistent with other reports [
Surgical method and duration are considered important risk factors in PU development [
This study showed that with highly imbalanced medical data, random undersampling could provide good results in some cases despite its simplicity. In fact, only four models generated using random undersubsampling had low sensitivity.
It is worth mentioning that no special data collection was required for this work. Collection of the data used is required at all Japanese acute care hospitals under the Japanese medical insurance system for purposes of medical fee reimbursement. Notably, this work allowed for the use of existing HIS data.
In this study, we investigated the possibility of predicting PU development in hospitalized patients from data collected by nurses on admission. Identifying the probability of developing PUs on patient admission enables nurses to take precautionary measures, and thus reduce the total number of in-hospital PU cases. The model uses the total NNS score combined with patient age, gender, and estimated surgical duration as predictive factors. Data were highly imbalanced, and the random undersampling method was used to overcome this problem. The logistic regression model achieved an average of 69.2% (6920/100) sensitivity and 82.8% (8280/100) specificity, demonstrating that random undersampling was effective for balancing the training dataset.
This study has a number of limitations worth noting. First, we used retrospective data with class imbalance. We are planning to investigate the results on new data. Second, the retrospective analysis was conducted in an acute care hospital setting, and thus cannot be generalized to long-term care hospitals. The current NNS does not reflect other important medical care tasks such as patient education, admission, and discharge instructions, coping with a dementia patient, fall prevention, and medication management. These are expected in a future NNS revision, and are required for further data mining of PU risk factors.
area under receiver operating characteristic curve
hospital information system
Nursing Needs Score
National Pressure Ulcer Advisory Panel
pressure ulcers
This study received no support from any funding agency in the public, commercial, or nonprofit sectors. The authors thank Emeritus Professor Hideki Nakanishi (Department of Plastic and Reconstructive Surgery, University of Tokushima Graduate School), Associate Professor Hirokazu Uemura (Department of Preventive Medicine, University of Tokushima Graduate School), Director Kikue Kida and Vice Director Sachiko Kondo (Department of Nursing, Tokushima University Hospital) for their cooperation, and Mr Hachiro Nakagawa for technical assistance.
None declared.