This is an openaccess article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.
Cardiac arrest is the most serious deathrelated event in intensive care units (ICUs), but it is not easily predicted because of the complex and timedependent data characteristics of intensive care patients. Given the complexity and time dependence of ICU data, deep learning–based methods are expected to provide a good foundation for developing risk prediction models based on large clinical records.
This study aimed to implement a deep learning model that estimates the distribution of cardiac arrest risk probability over time based on clinical data and assesses its potential.
A retrospective study of 759 ICU patients was conducted between January 2013 and July 2015. A characterlevel gated recurrent unit with a Weibull distribution algorithm was used to develop a realtime prediction model. Fivefold crossvalidation testing (training set: 80% and validation set: 20%) determined the consistency of model accuracy. The timedependent area under the curve (TAUC) was analyzed based on the aggregation of 5 validation sets.
The TAUCs of the implemented model were 0.963, 0.942, 0.917, 0.875, 0.850, 0.842, and 0.761 before cardiac arrest at 1, 8, 16, 24, 32, 40, and 48 hours, respectively. The sensitivity was between 0.846 and 0.909, and specificity was between 0.923 and 0.946. The distribution of risk between the cardiac arrest group and the non–cardiac arrest group was generally different, and the difference rapidly increased as the time left until cardiac arrest reduced.
A deep learning model for forecasting cardiac arrest was implemented and tested by considering the cumulative and fluctuating effects of timedependent clinical data gathered from a large medical center. This realtime prediction model is expected to improve patient’s care by allowing early intervention in patients at high risk of unexpected cardiac arrests.
Inhospital cardiac arrest (IHCA) is etiologically different from outofhospital cardiac arrest owing to the variety of underlying illnesses in hospitalized patients. Unfortunately, despite efforts to improve survival following IHCA, outcomes have not significantly improved over the last few decades [
Several studies have reported that mortality after IHCA was associated with the timing of cardiac arrest (day vs night shift), type of institution (teaching vs nonteaching hospital), and subsets of patients (ie, age and sex of patients) [
Given the complexity and time dependency of ICU data, machine learning–based methods including the deep learning–based early warning system and gradient boosting machine have provided a good basis to develop risk prediction models using large clinical data contained within electronic medical records [
This study aimed to develop a realtime deep learning model to predict the risk of cardiac arrest in critically ill patients in a medical intensive care unit (MICU). Then, we evaluated the performance of this system depending on the remaining time from the event occurrence.
We conducted a retrospective study with patients admitted to the MICU at the Asan Medical Center in Seoul, South Korea, between January 1, 2013, and July 31, 2015. For the development of a deep learning–based prediction model of cardiac arrest in critical ill patients of the MICU, we identified 759 distinct patients aged 18 years or older who stayed in the MICU for 1 day or more (
The data were preprocessed in 2 ways. First, we selected features that patients have in common (see Feature selection in
This study was approved by the institutional review board of the Asan Medical Center, Korea (institutional review board number 20151015). The need for informed consent was waived by the ethics committee as this study involved routinely collected medical data that were anonymously managed at all stages, including data cleaning and statistical analyses.
Data preprocessing flowchart. Obs: observation; TTE: time to event.
Characterlevel gated recurrent unit structure combined with the Weibull distribution.
The Weibull distribution, a continuous probability distribution, is a parametric model that can calculate the distribution form of survival time. Given the advantage of parametric models in survival analysis, the Weibull model is often used to estimate failure rate over time [
The probability density function of a Weibull random variable. k: shape parameter; λ: scale parameter; x: the quantity of time to failure.
The distribution consists of 2 parameters—the shape parameter
The challenging point of learning the model was related to the censoring feature of the data structure (ie, 1=cardiac arrest occurred or 0=censored). The TTE of cardiac arrest is actually observed data, unlike in the case where the data point is not censored. However, the TTE of cardiac arrest is unknown when the data point is censored. In this study, τ was defined as a threshold value indicating the time to safety in the censored group. We set 72 hours as a threshold based on the median number of hours that patients stayed in the MICU.
The outcomes of CharGRU with the Weibull distribution algorithm are 2 parameters corresponding to the shape and scale of the Weibull model. These 2 parameters enable calculation of likelihood through the function proposed in
The total number of patients was 759, consisting of 37 cardiac arrest patients and 722 non–cardiac arrest patients. As 45 variables for 1 patient are repeatedly observed 48 times, the number of observations for cardiac arrest patients is 1776 and that for censored patients is 34,656. Thus, the shape of the input data delivered to the GRU algorithm is a 3dimensional array of 36,432 × 48 × 45. If 45 variables of a timewise vector are missing, we apply a masking layer that skips the vector and the learning. It is then delivered sequentially to a layer of 50 GRU units. The activation function of this layer is an all hyperbolic tangent function. Next, a fully connected layer of 20 units is connected with the hyperbolic activation function. Finally, the 2 fully connected layers are connected to estimate the shape and scale, the parameters of the Weibull distribution with a softplus activation and exponential function, respectively.
A fivefold crossvalidation test (training set: 80% and validation set: 20%) was implemented to determine the consistency of the model’s accuracy. Overall, 5 models were learned independently from each dataset each time. Timedependent receiver operating characteristic (ROC) analysis was performed from the aggregated set of the probability of an individual having cardiac arrest in each time step, which was estimated from 5 validation sets [
All procedures for data preprocessing and model implementation were conducted through the open source programming languages R and Python. To handle data in the format of a data frame (ie, data table) and an array, 2 open source libraries—Pandas and Numpy—were used. CharGRU with a Weibull distribution was implemented in Keras (version 2.2.2), a wrapper library from Tensorflow (version 1.10.0), and a representative open source tool supporting the implementation of deep learning algorithms. Detailed concepts and mechanisms at the code level of this algorithm have been well documented in a previous study [
A total of 759 patients admitted in the ICU of the Asan Medical Center from March 2015 to March 2017 were enrolled in the study. Descriptive analysis was performed in 2 broad categories: demographics with 3 variables and diagnostic status with 8 variables. The Student
Descriptive statistics of the demographics and underlying diseases of the patients.
Variables  Cardiac group (n=37)  Censored group (n=722)  



Age (years), mean (SD)  62.509 (12.311)  60.526 (13.991)  <.001 ( 


Weight (kg), mean (SD)  59.734 (13.166)  57.816 (13.435)  <.001 ( 



. 



Male  28  451 




Female  9  271 










Yes  8  105 




No  29  617 








Yes  8  111 




No  29  611 








Yes  2  28 




No  35  694 








Yes  0  10 




No  37  712 








Yes  4  61 




No  33  661 








Yes  12  218 




No  25  504 








Yes  0  18 




No  37  704 








Yes  3  76 




No  34  646 

^{a}The digits outside the parentheses mean
As 5 crossvalidation procedures were performed in this study, each of the 5 models was trained independently.
Overall, 5 timedependent areas under the curve (TAUCs) were calculated using the aggregated set of 5 validation sets (
Results of timedependent receiver operating characteristic analysis according to the fold change. AUC: area under the curve.
Risk probability comparison between cardiac arrest and the non–cardiac arrest groups. The xaxis represents the time point, and the yaxis represents the distribution of probability density values for cardiac arrest obtained for each patient corresponding to each time point.
An additional problem with arrest prediction is predicting when a cardiac arrest event will occur. A cumulative distribution function was derived through the shape and scale inferred by the model from each time point. Using the Weibull distribution parameter derived for the 48 time points, curves corresponding to cumulative distribution functions were drawn (A in
(A) Cumulative distribution function lines from the predicted time point to censoring time point for a patient with cardiac arrest at 48 time points; Each function line is colorcoded. (B) Predicted hours remaining until a patient has cardiac arrest; the yaxis was limited to less than 25 hours for readability. pTime: predicted time.
Conversely, the distribution of the cumulative distribution function of a certain patient without cardiac arrest shows that, at all time points, the probability does not increase over time (A in
(A) Cumulative distribution function lines from the predicted time point to censoring time point for a patient without cardiac arrest at 48 time points; Each function line is colorcoded. (B) Predicted hours remaining until a patient has cardiac arrest; the yaxis was limited to less than 25 hours for readability. pTime: predicted time.
In this study, we developed the prediction model for cardiac arrest in critically ill patients through machine learning using electronic medical records. Besides vital sign, we used the underlying disease, laboratory data, medication, and organ failure as parameters to improve the accuracy of the prediction model. The TAUCs for TTE of 8, 16, and 24 hours were 0.942, 0.91, and 0.811, respectively, and the model performance decreased in accordance with increasing TTE.
In previous studies related to cardiac arrest predictions, modeling techniques that predict whether an event occurs within a predefined predicted time window have often been implemented [
The early recognition of cardiac arrest and its prompt correction are critical to reducing the mortality of critically ill patients. To decide clinically who is unstable or who is going to deteriorate, many intensivists often scrutinize the vital signs of intensive care patients, such as blood pressure, heart rate, respiratory rate, and peripheral capillary oxygen saturation [
As cumulative and fluctuating effects of clinical variables over time can be reflected in deep learning algorithms, the use of long time series data to predict cardiac arrest is ideal. However, it is not appropriate to take no action until the patient has accumulated sufficient time series data. Waiting for sufficient time (ie, 48 hours) to accumulate patient time series data in clinical settings is undesirable for both patients and intensivists. Even if variables have not yet accumulated for a sufficient amount of time, a model should be available. In this situation, the CharGRU structure allows the model to use the clinical variables. Specifically, the CharGRU model can predict the risk of a patient’s cardiac arrest using clinical variables accumulated up to the present time (ie, 3 hours after entering the ICU) [
The early detection of disease onset is challenging in terms of the configuration of deep learning algorithm structures and data pipelines, as there is no reference for
This study has limitations that need to be addressed in further studies before applying CharGRU with the Weibull distribution algorithm to clinics. In this study, rigorous validation was not performed while focusing on algorithm implementation using clinical data. As clinical data from only 1 medical institution were used, various additional validations are needed to generalize the results. To conduct rigorous validation, it is recommended to validate deep learning–based Weibull models using published data such as the modified early warning score [
Another limitation is the inability to fully control the reflection of certain effects in the collected data, which may affect the model results. For instance, data from a treated patient who is perceived to be in a very dangerous condition may cause a bias against the time series characteristics in the highrisk group [
The cardiac arrest survival rate in hospitals is about 24%, and even after survival, patients suffer from fatal problems such as brain damage [
Supplementary figures and tables.
characterlevel gated recurrent unit
gated recurrent unit
intensive care unit
inhospital cardiac arrest
interquartile range
medical intensive care unit
receiver operating characteristic
timedependent area under the curve
time to event
This study received support from the Asan Institute for Life Science, Asan Medical Center, Seoul, Republic of Korea (grant number 2017502), the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI19C1015) and the Bio & Medical Technology Development Program of the National Research Foundation (NRF) funded by the Korean government (MSIT; number NRF2019M3E5D4064682).
None declared.