This is an openaccess article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on https://medinform.jmir.org/, as well as this copyright and license information must be included.
Intraoperative hypotension has an adverse impact on postoperative outcomes. However, it is difficult to predict and treat intraoperative hypotension in advance according to individual clinical parameters.
The aim of this study was to develop a prediction model to forecast 5minute intraoperative hypotension based on the weighted average ensemble of individual neural networks, utilizing the biosignals recorded during noncardiac surgery.
In this retrospective observational study, arterial waveforms were recorded during noncardiac operations performed between August 2016 and December 2019, at Seoul National University Hospital, Seoul, South Korea. We analyzed the arterial waveforms from the big data in the VitalDB repository of electronic health records. We defined 2s hypotension as the moving average of arterial pressure under 65 mmHg for 2 seconds, and intraoperative hypotensive events were defined when the 2s hypotension lasted for at least 60 seconds. We developed an artificial intelligence–enabled process, named shortterm event prediction in the operating room (STEPOP), for predicting shortterm intraoperative hypotension.
The study was performed on 18,813 subjects undergoing noncardiac surgeries. Deeplearning algorithms (convolutional neural network [CNN] and recurrent neural network [RNN]) using raw waveforms as input showed greater area under the precisionrecall curve (AUPRC) scores (0.698, 95% CI 0.6900.705 and 0.706, 95% CI 0.6980.715, respectively) than that of the logistic regression algorithm (0.673, 95% CI 0.6650.682). STEPOP performed better and had greater AUPRC values than those of the RNN and CNN algorithms (0.716, 95% CI 0.7080.723).
We developed STEPOP as a weighted average of deeplearning models. STEPOP predicts intraoperative hypotension more accurately than the CNN, RNN, and logistic regression models.
ClinicalTrials.gov NCT02914444; https://clinicaltrials.gov/ct2/show/NCT02914444.
Intraoperative hypotension due to low blood pressure during surgery may cause acute kidney injury, myocardial injury, and mortality [
Researchers have utilized various statistical methods, machine learning, and deeplearning techniques to predict hypotension [
Realtime automated data acquisition of multiple biosignals in the OR has facilitated the implementation of various deeplearning technologies to predict intraoperative events. For example, invasive arterial waveformbased convolutional neural network (CNN) has yielded remarkable results in intraoperative hypotension prediction [
A CNN consists of convolution layers and pooling layers; convolution layers filter input data to produce feature maps indicating the locations and strength of detected features in the input data, and pooling layers downsample the feature maps by summarizing the presence of features in patches of the feature map [
The logistic regression model has been outperformed by deeplearning models in terms of various medical applications, including inhospital cardiac arrest prediction [
In this study, we propose the shortterm event prediction in the operating room (STEPOP) hypotension prediction system based on the weighted average ensemble of individual neural networks that utilizes biosignals recorded during noncardiac surgery. To this end, the arterial waveforms of 18,813 patients were selected, segmented, and labeled autonomously according to a criterion that enabled the construction and extension of deeplearning models with big data from realtime recording systems.
STEPOP was developed to predict intraoperative hypotension 5 minutes before it occurs based on big data from the VitalDB [
The process flow of STEPOP consists of (i) patient selection, (ii) data construction with automatic segmentation of biosignals and data cleaning, (iii) automatic labeling, and (iv) construction of the prediction model (
Process flow and criteria of shortterm event prediction in the operating room (STEPOP) for constructing the prediction model of intraoperative hypotension using VitalDB. CNN: convolutional neural network; RNN: recurrent neural network; NaN: missing values.
We selected all patients whose arterial waveforms were recorded during noncardiac operations performed between August 2016 and December 2019, at Seoul National University Hospital, Seoul, South Korea. A total of 21,321 patients were enrolled in this retrospective study for constructing the prediction model of intraoperative hypotension. The study was approved by the institutional review board of Seoul National University Hospital (H20081751152) and is registered at ClinicalTrials.gov (NCT02914444).
The arterial waveforms were recorded at 100 or 500 Hz and were downsampled to 100 Hz. Each 60second segment was observed paired with a 20second segment that occurred 5 minutes previously.
To detect artifacts in the arterial waveforms, we excluded waveforms clearly beyond the physiological range according to the following criteria: (1) segments with missing values, (2) segments with blood pressure over 200 mmHg or under 20 mmHg, (3) segments with a difference between the maximum and minimum pressure value under 20 mmHg, and (4) segments with a difference between adjacent values over 30 mmHg (pressure gradient over 3000 mmHg/second). The 20 or 60second segment of the arterial waveforms that met any of the criteria listed above was excluded from the dataset. No modifications were made to the extracted waveform segments.
Among the 21,321 patients, 2508 patients were excluded from the study after failing the data cleaning step according to the criteria. In total, the data segmentation process produced 2,041,805 segments from 18,813 patients. Patients were randomly split into 70/30 training and validation sets. Further, 1,428,553 segments from 13,178 patients’ data were used for algorithm development, and 613,252 segments from 5635 patients’ data were used for internal validation (
CONSORT diagram with flow of data construction.
STEPOP predicts hypotension 5 minutes before its onset based on 20second arterial waveforms. First, we defined 2s hypotension as the moving average of arterial pressure under 65 mmHg for 2 seconds, and the intraoperative hypotensive event was defined as the case in which the 2s hypotension lasts for at least 60 seconds. Accordingly, 20second segments of the arterial waveform 5 minutes before the event were selected and labeled “positive instances.” If the 2second moving average was maintained over 75 mmHg for at least 60 seconds, it would be considered a nonhypotensive event. The 20second segments 5 minutes before the onset were selected and labeled “negative.”
We developed an ensemble average model from two distinct deeplearning layers of a CNN and RNN. The combination of multiple neural networks can outperform individual networks while offering the advantage of generalization [
The final stage of model construction computes the ensemble average prediction value,
Shortterm event prediction in the operating room (STEPOP) model construction. K×F denotes the kernel size and number of filters. ReLU activation was used after each convolution layer, and the sigmoid was used for the final activation. Normalization, pooling, and dropout layers are omitted in the figure. LSTM: long shortterm memory; FC: fully connected layer.
We used the Pytorch deeplearning framework [
We evaluated the performance of the proposed model by comparing it with the logistic regression model based on the feature set of 12 features from the 20second arterial waveforms (
Features from the arterial waveform segments.
Feature symbol  Description 
Mean_beat_length  Average time of cardiac cycle 
MAP^{a}  Average MAP of cardiac cycle 
PP^{b}_max  Maximum value among pulse pressure 
PP_min  Minimum value among pulse pressure 
PP_range  PP_max–PP_min 
PP_avg  Average pulse pressure 
PPV^{c}  (PP_max–PP_min)×2.0/(PP_max+PP_min) 
Systolic_time_avg  Average systolic time 
Systolic_pressure_avg  Average systolic pressure 
Systolic_pressure_range  Difference between maximum systolic pressure and minimum systolic pressure 
Diastolic_pressure_avg  Average diastolic pressure 
Beat_area_avg  Average of area under cardiac cycles 
^{a}MAP: mean arterial pressure.
^{b}PP: pulse pressure.
^{c}PPV: pulse pressure variation.
We performed robust scaling after extracting the features, as
The logistic regression model with fivefold crossvalidation was implemented using scikitlearn [
The prediction models of an imbalanced dataset are evaluated in terms of the performance metrics AUPRC, area under the receiver operating characteristic curve, precision, and sensitivity (recall) since the negative data points significantly outnumbered positive data points [
The proposed method helped us to select 18,813 patients for the study. The mean age of the group was 58.5 (SD 15.3) years. Approximately 49.3% of patients in the group were male. The training cohort (n=13,178) presented 1,373,378 negative segments and 55,175 positive segments (total 476,184 minutes). The validation cohort (n=5635) presented 587,413 negative segments and 25,839 positive segments (total 204,417 minutes).
Study population characteristics.
Characteristic  Total  Training cohort  Validation cohort  
Number of patients  18,813  13,178  5635  N/A^{a}  
Age (years), mean (SD)  58.5 (15.3)  58.6 (15.2)  58.2 (15.4)  .13  
Weight (kg), mean (SD)  63.4 (12.8)  63.5 (12.8)  63.4 (12.8)  .68  
Height (cm), mean (SD)  162.2 (10.0)  162.2 (10.0)  162.4 (9.9)  .07  
Male, n (%)  9270 (49.27)  6416 (48.69)  2854 (50.65)  .01  




.47  

I  4352 (23.14)  3084 (23.40)  1268 (22.50) 


II  11,428 (60.75)  7970 (60.48)  3458 (61.37) 


III  2824 (15.01)  1980 (15.03)  844 (14.98) 


IV  196 (1.04)  137 (1.04)  59 (1.05) 


IV  13 (0.07)  7 (0.05)  6 (0.11) 

^{a}N/A: not applicable.
^{b}ASA: American Society of Anesthesiologists.
Feature characteristics.
Feature symbol  Positive event, mean (SD)  Negative event, mean (SD)  
Mean_beat_length (s)  0.82 (0.19)  0.89 (0.17)  <.001 
MAP^{a} (mmHg)  64 (12)  90 (12)  <.001 
PP^{b}_max (mmHg)  55 (16)  60 (15)  <.001 
PP_min (mmHg)  45 (16)  50 (14)  <.001 
PP_range (mmHg)  10.0 (10.7)  9.8 (9.8)  <.001 
PP_avg (mmHg)  50 (15)  55 (14)  <.001 
PPV^{c}  0.23 (0.29)  0.20 (0.25)  <.001 
Systolic_time_avg (s)  0.13 (0.04)  0.14 (0.04)  <.001 
Systolic_pressure_avg (mmHg)  98 (19)  125 (18)  <.001 
Systolic_pressure_range (mmHg)  10.7 (10.2)  10.4 (9.0)  <.001 
Diastolic_pressure_avg (mmHg)  49 (10)  71 (11)  <.001 
Beat_area_avg (mmHg×s)  53 (16)  79 (18)  <.001 
^{a}MAP: mean arterial pressure.
^{b}PP: pulse pressure.
^{c}PPV: pulse pressure variation.
(A) Optimal weight value α on 10% of the training set. (B) Precisionrecall curve of developed models. AUPRC: area under the precisionrecall curve; CNN: convolutional neural network; RNN: recurrent neural network; STEPOP: shortterm event prediction in the operating room.
Performance of each algorithm in the internal validation cohort.
Algorithm  AUPRC^{a} (95% CI)  AUROC^{b} (95% CI)  Sensitivity^{c} (95% CI)  Precision^{c} (95% CI)  



0.6  0.7  0.8  0.6  0.7  0.8  
STEPOP^{d}  0.716 
0.961 
0.600 
0.700 
0.800 
0.742 
0.647 
0.502 

Convolutional neural network  0.698 
0.955 
0.600 
0.700 
0.800 
0.717 
0.615 
0.466 

Recurrent neural network  0.706 
0.958 
0.600 
0.700 
0.800 
0.738 
0.639 
0.488 

Logistic regression  0.673 
0.948 
0.600 
0.700 
0.800 
0.711 
0.622 
0.481 
^{a}AUPRC: area under the precisionrecall curve.
^{b}AUROC: area under the receiver operating characteristic curve.
^{c}Sensitivity and precision values were evaluated at the thresholds for sensitivity of 0.6, 0.7, and 0.8.
^{d}STEPOP: shortterm event prediction in the operating room.
Example of a patient record depicting the arterial pressure and STEPOP prediction values over time. Arterial pressure denotes the 2second moving average of the arterial pressure. STEPOP: shortterm event prediction in the operating room.
In this retrospective observational study, we developed deeplearning and machinelearning algorithms to predict an intraoperative hypotension event 5 minutes before its onset by leveraging a big data repository from an automatic recording system in the OR. Processing big data introduces several methodological challenges and opportunities in medical research [
This study extends previous work on the HPI using an identical input of the highfidelity 20second arterial waveform. The HPI is the only algorithm currently used for predicting intraoperative hypotension [
Deeplearning algorithms may detect subtle changes in the arterial waveform, which predict sudden drops in the arterial pressure. These changes are likely to be masked or diminished when represented as features. As shown in
Finally, the ensemble average of CNN and RNN predicted hypotension more accurately than each deeplearning model. In this study, the optimal weights for the ensemble of LSTM and CNN outputs were 0.65 and 0.35, respectively. This showcases the improved intraoperative hypotension prediction by the hybrid model STEPOP over a single deeplearning model or logistic regression model.
This approach has a few limitations. First, we defined hypotension arbitrarily (2second pressure moving average under 65 mmHg for hypotensive events, and 2second moving average over 75 mmHg for nonhypotensive events). Prospective research must be performed to observe the effect of these criteria on the performance of the algorithms. Second, although a relatively large (N>10,000) cohort of patient data was used, it was retrieved from a single database. Future research will include external validations of different population distributions and settings. Finally, the threshold values and corresponding response of clinicians according to the STEPOP prediction value must be determined for practical use in the OR. Prospective studies in actual clinical practice are needed to solve these limitations.
We developed STEPOP utilizing a big data repository and constructed a prediction model of shortterm intraoperative hypotension. The weighted average of the deeplearning models performed the best in the prediction of hypotension. The proposed algorithms use only the 20second arterial waveform without requiring separate feature computations. Consequently, they can be easily implemented in scenarios with the possibility of invasive blood pressure monitoring and can replace the HPI algorithm in those situations. The proposed solution can be extended and practically used for the realtime prediction of adverse events in the OR or intensive care units. This in turn is expected to improve clinical outcomes and reduce the burden of medical staff.
area under the precisionrecall curve
convolutional neural network
hypotension prediction index
long shortterm memory network
operating room
recurrent neural network
shortterm event prediction in the operating room
This research was supported by a grant funded by the Ministry of Science and ICT (NRF2020R1A2C1013152) of the Republic of Korea.
None declared.