This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.
Electronic recording of vital sign observations (e-Obs) has become increasingly prevalent in hospital care. The evidence of clinical impact for these systems is mixed.
The objective of our study was to assess the effect of e-Obs versus paper documentation (paper) on length of stay (time between trauma unit admission and “fit to discharge”) for trauma patients.
A single-center, randomized stepped-wedge study of e-Obs against paper was conducted in two 26-bed trauma wards at a medium-sized UK teaching hospital. Randomization of the phased intervention order to 12 study areas was computer generated. The primary outcome was length of stay.
A total of 1232 patient episodes were randomized (paper: 628, e-Obs: 604). There were 37 deaths in hospital: 21 in the paper arm and 16 in the e-Obs arm. For discharged patients, the median length of stay was 5.4 (range: 0.2-79.0) days on the paper arm and 5.6 (range: 0.1-236.7) days on the e-Obs arm. Competing risks regression analysis for time to discharge showed no difference between the treatment arms (subhazard ratio: 1.05; 95% CI 0.82-1.35;
The phased introduction of an e-Obs documentation system was not associated with a change in length of stay. A greater proportion of patient episodes contained an EWS≥3 using the e-Obs system, but this was not associated with a change in “escalation time.”
ISRCTN Registry ISRCTN91040762; http://www.isrctn.com/ISRCTN91040762 (Archived by WebCite at http://www.webcitation.org/72prakGTU)
Physiological vital signs are regularly measured in hospitalized patients. Deterioration in vital signs often precedes adverse outcomes [
Until recently, vital signs and EWS have been recorded on paper observation charts. The shortcomings of paper charts include incorrect score assignment to the vital signs and incorrect calculation of EWS [
In this study, we prospectively assessed whether the deployment of the VitalPAC e-Obs system (VitalPAC; The Learning Clinic), compared with the paper-based system, changed patients’ hospital length of stay (since the conclusion of this study, The Learning Clinic has been acquired by System C, Maidstone, Kent). Our null hypothesis was that length of stay remained unaltered. A cluster-randomized design was not appropriate as the VitalPAC e-Obs system was to be introduced gradually in the trauma unit. We, therefore, evaluated the intervention as it was being introduced using a stepped-wedge study design.
We conducted a randomized stepped-wedge interventional study in the two adult inpatient wards of the trauma unit at the John Radcliffe Hospital, Oxford University Hospitals National Health Service Trust. Each ward had 26 inpatient beds. A stepped-wedge study is one in which the intervention is phased into the study population within clusters across successive time periods, with the time being determined by randomization [
Initially, nursing staff measured vital signs (blood pressure, pulse, oxygen saturation, temperature, respiratory rate, and consciousness level) using spot-check monitors and documented the result on paper. Nurses manually calculated the EWS and recorded the score on paper. The EWS used was the previously published centile-EWS [
The intervention was the VitalPAC e-Obs system, which allowed vital signs to be documented on a hand-held device. The EWS was then automatically calculated, and relevant hospital guidance for escalation was displayed. EWS were also displayed at a central patient ward list that adjusted colors and symbols to prompt nurses to record timely observations according to the hospital protocol.
Prior to study commencement, VitalPAC was installed and tested and staff were trained to use the system. Refresher teaching was also provided for the paper system, and staff were reminded of local clinical escalation policy. During the training period, the paper system was used throughout the trauma unit, but staff had the opportunity to enter data into a test installation of VitalPAC. Study staff attended the ward daily during the week and once at weekends, providing top-up training throughout the duration of the study.
The e-Obs intervention was phased in 12 clusters. Each cluster was a physical zone within the trauma unit consisting of a 4-bed bay or a collection of six 1-bed side rooms. The clusters received the electronic intervention sequentially. A new zone switched from the control to the intervention every 3 weeks on Tuesday at 2 pm.
All episodes from patients aged ≥16 years admitted to the trauma unit during the study period were considered for eligibility. Episodes were excluded from analysis if the treatment plan for the patient was palliative at admission to the ward.
Clinical and demographic data on study participants were obtained from an electronic patient record (
We also recorded initial ward and zone on the trauma unit; date and time (hh:mm) of admission into the ward; date, time (hh:mm), and clinician-recorded EWS for the first vital sign observations recorded on the trauma unit; total number of vital sign observations on the unit recorded electronically and on paper; “fit to discharge” date; actual hospital discharge date and time (hh:mm); in-hospital mortality and 30-day mortality following ward admission; and unplanned admission to the intensive care unit (ICU) or a cardiac arrest for each episode.
Two other electronic sources,
The primary endpoint was length of stay (the time from admission to the trauma unit until “fit to discharge”). “Fit to discharge” was defined as the first of the following: discharged from the ward to home or alternative care or accepted by social services as a “delayed discharge.” This outcome measure was chosen because some trauma patients were known to have extended stays while waiting for suitable support mechanisms to be put in place outside hospital. Secondary endpoints were mortality (in-hospital and 30-day following ward admission), whether a patient experienced a cardiac arrest or an unplanned ICU admission; the time between observations (length of stay/total number of observations); the time until a patient first scored EWS≥3; and the time between the first observation that scored EWS≥3 and the subsequent observation (“escalation time”).
The number of participants was determined by the speed at which zones transitioned from paper to e-Obs. Slower transitions would include more participants for each step, thereby increasing the power of the study. However, clinical staff wished to minimize the concurrent use of multiple systems. Therefore, a clinically accepted transition rate of 1 zone every 3 weeks was chosen, and no sample size calculation was undertaken.
A random sequence generated using MATLAB (function
All zones were recruited and enrolled at baseline and followed for the entire duration of the study. Research nurses administered the sequential assignment to the e-Obs intervention. The nurses visited the ward during each transition to facilitate adherence to the change from paper to e-Obs.
Patients were allocated to either paper or electronic recording of observations based on the zone to which they were allocated on arrival to the trauma unit. Patients remained with the same recording method even if subsequently moved. Allocation to the initial zone was determined by normal ward practices, which remained unchanged during the study period. Therefore, the allocation ratio could not be determined a priori.
The randomization sequence was concealed from patients, clinical staff, and all researchers involved in data collection until the day a new zone was due to receive e-Obs. It was not possible to conceal the intervention.
Episode characteristics, including patient age and length of stay, were summarized by study arm. A time-to-event analysis, Cox proportional hazards regression, was undertaken for length of stay with the competing risk of death in hospital, for death in hospital with discharge from hospital as a competing risk, and for time to first EWS≥3. The intervention arm, step, age, and sex of patients were included as covariates. Subhazard ratios were calculated with respect to paper charts, age>80 years, and male sex. The step was included as a continuous measure of time and as a factor with 13 levels. The study design has two levels, patient and study zone. The SEs of the coefficients in the Cox proportional hazards model were adjusted for the cluster variable, study zone. A similar competing risks Cox regression analysis was undertaken for “escalation time,” censored at 200 hours.
The binary outcome, death within 30 days of admission, was analyzed using a logistic regression model and the SEs of the coefficients were adjusted for the cluster variable, study zone. The numbers of cardiac arrests and admissions to ICU were reported.
All analyses were performed on an intention-to-treat basis, with patients analyzed according to the randomization intervention. Furthermore, all analyses were completed for hospital episodes. A patient may have had multiple episodes consisting of distinct admissions to the study wards for unrelated reasons. We assumed a priori that the number of such admissions was small such that each episode may be treated as an independent event. Post hoc, we repeated the analyses on the per-protocol populations.
All statistical analyses were performed using Stata (Stata Statistical Software: Release 14. College Station, TX; StataCorp LP) [
A total of 1518 admissions to the trauma unit were recorded between August 31, 2011 and May 31, 2012. After excluding 286 episodes that did not meet the study criteria or for which patients had declined consent, 1232 episodes (from 1199 patients) were included for analysis (
Of the included episodes, 628 were randomized to paper and 604 to e-Obs. Moreover, 873 episodes (paper: 558, e-Obs: 315) had vital sign observations that were fully consistent with the randomized intervention. A further 194 episodes (paper: 32, e-Obs: 162) had over 80% of observations on the allocated intervention. Vital sign observation charts were absent from the paper notes for 13 episodes.
Baseline characteristics of the study patients are shown in
There were 37 deaths in hospital: 21 in the paper arm and 16 in the e-Obs arm. For patients who were discharged, the median length of stay (time from admission to “fit to discharge”) was 5.4 (range: 0.2-79.0) days on the paper arm (607 patients) and 5.6 (range: 0.1-236.7) days on the e-Obs arm (588 patients). Longer time to discharge was associated with greater age, but there was no difference between the treatment arms (
Flow chart for patients’ enrollment in the study. “Escalation Time” is the time between the first observation that scored Early Warning Score (EWS)≥3 and the subsequent observation. ICU: intensive care unit.
Results from the competing risks Cox regression analysis for time to death in hospital with discharge from hospital as a competing risk are reported in
There were 45 deaths within 30 days from admission to hospital: 23 in the paper arm and 22 in the e-Obs arm. The results from the logistic regression analysis of the number of deaths within 30 days of admission are reported in
There were 8 admissions to the ICU from the paper arm and 7 from the e-Obs arm, and there were 2 cardiac arrests in each arm. The median time between observations for those without a delayed discharge was 7.1 (interquartile range [IQR]: 5.0-9.8) hours on paper and 7.0 (IQR: 5.3-9.5) hours on e-Obs.
All results presented here are for intention-to-treat analysis. The results of the per-protocol analysis are shown in
Patient baseline characteristics (N=1232 episodes).
Characteristic | Paper (n=628) | e-Obsa (n=604) | |
Age (years) mean (SD) | 58.1 (23.4) | 60.4 (23.3) | |
Males, n (%) | 308 (49.0) | 316 (52.3) | |
Ward 1:Ward 2 | 312:316 | 280:324 | |
White British | 471 (74.9) | 446 (73.6) | |
Not stated | 110 (17.5) | 129 (21.3) | |
Other | 48 (7.6) | 31 (5.1) | |
Lower limb (excluding neck of femur) | 179 (28.5) | 188 (31.1) | |
Neck of femur | 160 (25.5) | 169 (28.0) | |
Upper limb (excluding wrist) | 59 (9.4) | 63 (10.4) | |
Polytrauma (excluding head) | 48 (7.6) | 45 (7.5) | |
Wrist | 38 (6.1) | 21 (3.5) | |
Spinal trauma | 33 (5.3) | 30 (5.0) | |
Polytrauma + head | 29 (4.6) | 23 (3.8) | |
Nontrauma | 29 (4.6) | 15 (2.5) | |
Other | 53 (8.4) | 50 (8.3) | |
Trauma | 581 (92.5) | 574 (95.0) | |
Other | 47 (7.5) | 30 (4.9) | |
Emergency department (not via intensive care unit) | 323 (51.4) | 342 (56.3) | |
Trauma clinic within Oxford University Hospitals Trust | 108 (17.2) | 107 (17.6) | |
Other ward | 89 (14.3) | 57 (9.7) | |
Transfer from other United Kingdom hospital | 24 (3.8) | 24 (4.0) | |
Emergency department via emergency admissions unit | 50 (7.9) | 50 (8.2) | |
Other | 34 (5.4) | 24 (4.0) | |
1 | 101(16.1) | 97 (16.1) | |
2 | 97 (15.4) | 107 (17.7) | |
3 | 92 (14.6) | 86 (14.2) | |
4 | 18 (2.9) | 22 (3.7) | |
5 | 3 (0.5) | 1 (0.2) | |
1E-5E | 11 (1.8) | 12(2.0) | |
Unrecorded | 180 (28.7) | 169 (28.0) | |
Not applicable | 126 (20.0) | 110 (18.2) |
ae-Obs: electronic recording of vital sign observations.
Results from the competing risks regression analysis for time to discharge from hospital with death in hospital as a competing risk, from the competing risks regression analysis for time to death in hospital with discharge from hospital as a competing risk, and the logistic regression for number of deaths within 30 days from admission.
Variable | Time to discharge from hospital with death in hospital as a competing risk | Time to death in hospital with discharge from hospital as a competing risk | Number of deaths within 30 days from admission | |||
Subhazard ratio (95% CI) | Subhazard ratio (95% CI) | Odds ratio (95% CI) | ||||
e-Obsa | 1.05 (0.82-1.35) | .68 | 0.77 (0.42-1.40) | .39 | 0.82 (0.46-1.47) | .51 |
Step | 1.00 (0.97-1.03) | .99 | 0.98 (0.88-1.10) | .72 | 1.01 (0.92-1.12) | .80 |
Age ≥80 years | 0.62 (0.57-0.67) | <.001 | 5.69 (3.48-9.30) | <.001 | 8.93 (5.69-14.01) | <.001 |
Female | 1.09 (0.96-1.25) | .19 | 0.63 (0.35-1.12) | .11 | 0.60 (0.39-0.94) | .02 |
ae-Obs: electronic recording of vital sign observations.
Kaplan-Meier failure estimates for time (hours) from admission until first Early Warning Score (EWS)≥3. e-Obs: electronic recording of vital sign observations.
Kaplan-Meier failure estimates for time (hours) to next set of observations after the first Early Warning Score (EWS)≥3. e-Obs: electronic recording of vital sign observations.
To our knowledge, this is the first stepped-wedge evaluation of an e-Obs intervention. We found no difference in length of stay (the primary outcome) between paper and e-Obs. A significantly greater proportion of patient episodes contained an EWS≥3 using the e-Obs system than using paper. However, we found no difference in “escalation time.” Per-protocol analysis showed improved in-hospital survival with e-Obs, but this was not shown either on intention-to-treat analysis or for outcome at 30 days. The absence of intraclass correlation indicates that patient groups were well matched at each individual step level.
Although 86.61% (1067/1232) of patients had over 80% of their observations in the correct format, cross over was unequal. Only 79.0% (477/604) of patients in the electronic observations arm had more than 80% of their observations recorded electronically. The presence of multiple documentation systems in the wards may have led to confusion and suboptimal system use. To avoid this issue, future studies might ensure clear physical separation between zones to prevent study arm contamination. Our follow-up study over 4 hospitals instead considers whole wards as a study zone [
Length of stay is a complex outcome, risking confounding by competing interventions. However, it is an important outcome for patients and their families [
In this study, more patient episodes contained an EWS≥3 using the e-Obs system than paper. We have previously shown paper documentation errors to be biased toward values lower than the true EWS, particularly when a patient first develops physiological instability [
Despite differences in documented EWS, there was no difference in the timeliness of observations between the two cohorts. This supports the results reported by Hands et al, in which an e-Obs system was introduced to all adult inpatient areas of one hospital [
There was also no difference in the primary outcome, patient length of stay from admission to the trauma unit until “fit to discharge” after accounting for potential confounding variables. Previous evaluations of e-Obs systems have focused on process improvements such as data accuracy and speed of documentation [
A more recent study suggested substantial reductions in hospital mortality in two hospitals that implemented VitalPAC [
The previous before-and-after studies are inherently limited in their ability to account for temporal changes in covariates [
Although e-Obs had no effect on patient outcomes in our study, there are some positive findings. First, the frequency of observations before and after e-Obs remained stable. Furthermore, although the timeliness of observations when patients were physiologically unstable did not improve, they also did not worsen, matching previous reports of e-Obs introduction [
The results here are specific to the VitalPAC e-Obs system. This system contains some key features that are available in alternative e-Obs solutions; these include automatic EWS calculation and real-time ward lists [
The introduction of an electronic system for recording vital sign observations was not associated with reduction in time from admission to the trauma unit until “fit to discharge.” More patient episodes contained an EWS≥3 using the e-Obs system, but this was not associated with a change in “escalation time.”
Per-protocol analysis.
CONSORT‐EHEALTH checklist (V 1.6.1).
electronic recording of vital sign observations
Early Warning Score
intensive care unit
We thank the Learning Clinic (System C) for provision of the VitalPAC system and full training for staff during the study. We also thank Jane Cottrell for assistance in developing the study protocol and Jane Boutflower and Gloria Noreen for collecting data during the study. This work was supported by the Oxford National Institute for Health Research Biomedical Research Centre (DFRWAO00).
DCWW assisted in collecting the data, managed and refined the data, produced overview statistics, and prepared the manuscript draft. JK collected the data and assisted in preparing the manuscript. JB developed the statistical methodology, computed the statistical analysis of the data, and assisted in preparing the manuscript. LT and PJW designed the research study, provided overall supervision, and revised the manuscript. All authors have read and approved the manuscript.
PJW, DCWW, and LT developed an alternative e-Obs system (SEND) after study completion and prior to paper submission. In February 2017, this system was licensed by the University of Oxford and the Oxford University Hospitals National Health Service Trust to Drayson Technologies. Under the license agreement, PJW, DCWW, and LT may receive future royalties. DCWW and LT have undertaken consultancy for Drayson Technologies. JK and JB declare no conflict of interest. The Learning Clinic had no access to the data or influence on the analysis of the study.