This paper is in the following e-collection/theme issue:

Machine Learning (1465) Artificial Intelligence (1371) Depression and Mood Disorders; Suicide Prevention (1363)

Published on in Vol 8, No 7 (2020): July

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/14500, first published .
Identifying the Medical Lethality of Suicide Attempts Using Network Analysis and Deep Learning: Nationwide Study

Identifying the Medical Lethality of Suicide Attempts Using Network Analysis and Deep Learning: Nationwide Study

Identifying the Medical Lethality of Suicide Attempts Using Network Analysis and Deep Learning: Nationwide Study

Authors of this article:

Bora Kim1 Author Orcid Image ;   Younghoon Kim2 Author Orcid Image ;   C Hyung Keun Park3, 4 Author Orcid Image ;   Sang Jin Rhee3, 4 Author Orcid Image ;   Young Shin Kim1 Author Orcid Image ;   Bennett L Leventhal1 Author Orcid Image ;   Yong Min Ahn3, 4 Author Orcid Image ;   Hyojung Paik2 Author Orcid Image
Article Authors Cited by (3) Tweetations (8) Metrics

    Original Paper

    1Department of Psychiatry, University of California, San Francisco, San Francisco, CA, United States

    2Center for Supercomputing Applications, Division of Supercomputing, Korea Institute of Science and Technology Information (KISTI), Daejeon, Republic of Korea

    3Department of Neuropsychiatry, Seoul National University Hospital, Seoul, Republic of Korea

    4Department of Psychiatry and Behavioral Science, Seoul National University College of Medicine, Seoul, Republic of Korea

    *these authors contributed equally

    Corresponding Author:

    Hyojung Paik, PhD

    Center for Supercomputing Applications

    Division of Supercomputing

    Korea Institute of Science and Technology Information (KISTI)

    245 Daehak-ro

    Yuseong-gu

    Daejeon, 305-806

    Republic of Korea

    Phone: 1 82 42 869 1004

    Email: hyojungpaik@kisti.re.kr


    Background: Suicide is one of the leading causes of death among young and middle-aged people. However, little is understood about the behaviors leading up to actual suicide attempts and whether these behaviors are specific to the nature of suicide attempts.

    Objective: The goal of this study was to examine the clusters of behaviors antecedent to suicide attempts to determine if they could be used to assess the potential lethality of the attempt. To accomplish this goal, we developed a deep learning model using the relationships among behaviors antecedent to suicide attempts and the attempts themselves.

    Methods: This study used data from the Korea National Suicide Survey. We identified 1112 individuals who attempted suicide and completed a psychiatric evaluation in the emergency room. The 15-item Beck Suicide Intent Scale (SIS) was used for assessing antecedent behaviors, and the medical outcomes of the suicide attempts were measured by assessing lethality with the Columbia Suicide Severity Rating Scale (C-SSRS; lethal suicide attempt >3 and nonlethal attempt ≤3).

    Results: Using scores from the SIS, individuals who had lethal and nonlethal attempts comprised two different network nodes with the edges representing the relationships among nodes. Among the antecedent behaviors, the conception of a method’s lethality predicted suicidal behaviors with severe medical outcomes. The vectorized relationship values among the elements of antecedent behaviors in our deep learning model (E-GONet) increased performances, such as F1 and area under the precision-recall gain curve (AUPRG), for identifying lethal attempts (up to 3% for F1 and 32% for AUPRG), as compared with other models (mean F1: 0.81 for E-GONet, 0.78 for linear regression, and 0.80 for random forest; mean AUPRG: 0.73 for E-GONet, 0.41 for linear regression, and 0.69 for random forest).

    Conclusions: The relationships among behaviors antecedent to suicide attempts can be used to understand the suicidal intent of individuals and help identify the lethality of potential suicide attempts. Such a model may be useful in prioritizing cases for preventive intervention.

    JMIR Med Inform 2020;8(7):e14500

    doi:10.2196/14500

    Keywords

    suicide; deep learning; network; antecedent behaviors 


    Suicide is an important public health epidemic globally. The suicide incidence in the United States has increased in recent years from 10.9/100,000 in 2006 to 13.3/100,000 in 2015 [1], and nearly 45,000 Americans killed themselves in 2016 [2]. The suicide rate in South Korea is the highest among developed countries, and mortality attributable to suicide exceeds that attributable to common diseases, including diabetes, pneumonia, and liver disease [3]. Suicide is a preventable health problem, but effective prevention strategies are lacking because it is a complex issue, and thus, it is difficult for researchers to develop a cause and prediction model [4].

    The management of suicide attempts is an urgent clinical problem, and preventing further attempts is particularly important. The risk of suicide has many components, and of these, a previous suicide attempt is among the most important [5,6]. Understanding the nature of suicide attempts and possible associations with subsequent death by suicide may facilitate the design of interventions targeted at specific risk characteristics for particular individuals, thereby increasing clinical effectiveness and reducing morbidity and mortality in this high-risk population.

    Suicide attempts are highly heterogeneous and range from a “cry for help” to a nearly lethal attempt with self-mutilation and actual suicide [7]. In the present study, among the outcomes of suicide attempts, we consider the possible medical lethality of attempts as medical consequences, as well as the severity of the physical harm to individuals. Medical lethality as an outcome can be considered the degree of danger to life resulting from a suicide attempt [8]. In addition, most people who attempt suicide will communicate their intent in various forms before they actually attempt suicide [9]. However, it is unclear whether understanding the specific relationships with the behaviors leading up to actual suicide attempts can help to provide guidance for reducing suicide attempts. The relationship between the lethality of a preceding suicide attempt and medical lethality following a subsequent suicide attempt is unknown [10]. Thus, predictive models and explorations of the thought structures of individuals who attempt suicide are still lacking.

    We hypothesized that among individuals who attempt suicide, the relationships among their antecedent suicidal thoughts, behaviors, and communications will exhibit specific patterns, thereby allowing us to predict their future risk and lethality. A network model can be employed to conceptualize the complex dynamic systems comprising each interacting symptom [11-13]. Owing to this advantage of network analysis, previous studies have explored the nested interactions among the features of psychopathology [14] or the symptoms of major depressive disorder [15]. The results obtained by network-based analysis can successfully depict multiple nodes as variables, and multiple edges represent the mutual interactions between each pair of variables (ie, nodes). In this study, we employed network analysis to build a model where the nodes represent unique aspects of the expressed suicidal intent and the edges depict the correlations among these nodes.

    After determining the relationship values among the antecedent behaviors, we applied deep learning to identify the medical outcomes of subsequent suicide attempts. Deep learning is an emerging machine learning technique for predictive modeling in various applications, which is based on data observations but without domain-specific knowledge. The application of deep learning in the psychiatric field to develop Woebot, a text-based chatbot, has facilitated depression care [16]. However, the success of machine learning-based approaches has been limited in the identification of the central elements of suicidal intents and in prediction modeling based on the information collected by health care providers to meaningfully enhance clinical care. In this study, we employed a network-based method to explore the connections between communicative behaviors prior to suicide attempts with lethal or less lethal outcomes by using data that are routinely collected by physicians. Moreover, to train the complex connections between the antecedent behaviors, our deep learning model utilized the novel relationship values among suicidal intent elements.


    Study Sample

    We analyzed data obtained from the Korea National Suicide Survey [17], which was a nationwide multicenter study of subjects from two cohorts comprising individuals who attempted suicide and were recruited by retrospective chart review and those who attempted suicide and completed psychiatric evaluations by on-call psychiatric residents. The subjects from the second dataset were used in this study. All individuals who attempted suicide visited the emergency room (ER), and they were evaluated in semistandardized interviews at 17 medical centers across South Korea from May 1, 2013, to November 7, 2013. Deaths in the ER were excluded from the data. Among 1359 individuals who attempted suicide, 1112 were included in the final analysis after excluding missing data from the Columbia Suicide Severity Rating Scale (C-SSRS) and Beck Suicide Intent Scale (SIS) (Figure 1).

    ‎
    Figure 1. Study overview. C-SSRS: Columbia Suicide Severity Rating Scale; Edge: Association between a pair of nodes based on the weighted correlations according to graphical lasso; Node: Nodes for the measured elements of the SIS and C-SSRS fatality assessment; SIS: Beck Suicide Intent Scale.
    View this figure

    Outcome: Medical Lethality of Suicide Attempts

    The outcome of this study involving the medical lethality of suicide attempts was assessed by a clinician and classified based on the “actual lethality or medical damage” using the C-SSRS. The validated Korean version of the C-SSRS was used [18], and lethality was rated as follows: 1, no physical damage or very minor physical damage (eg, surface scratches); 2, minor physical damage (eg, lethargic speech and mild bleeding); 3, moderate physical damage (eg, conscious but sleepy); 4, moderately severe physical damage (eg, comatose with reflexes); 5, severe physical damage (eg, comatose without reflexes); and 6, death [19]. We used a lethality scale with the following two categories: score 3, a less lethal outcome of a suicide attempt and score >3, a lethal outcome of a suicide attempt.

    Suicidal Intents: Suicidal Intent Thoughts, Behaviors, and Communications

    The SIS was used to identify the elements of suicidal intent thoughts, behaviors, and communications [20]. The scale contains 15 questions (ie, SIS 1-15), and all of the items are scored on a scale from 0 to 2 for severity, where the total sum of the scores ranges from 0 to 30. In this study, we calibrated the SIS scale from 1 to 3 to calculate the relationships among the features. The SIS comprises the following two parts: objective circumstances of the attempt and the subject’s self-reported intentions and expectations regarding the attempt. The objective factors (SIS 1-8) are as follows: SIS 1, isolation; SIS 2, timing of intervention feasibility; SIS 3, active or passive precautions against discovery or intervention; SIS 4, acting to get help during or after the suicide attempt; SIS 5, final acts in anticipation of death; SIS 6, active preparation for the suicide attempt; SIS 7, suicide note; and SIS 8, overt communication of intent before the suicide attempt. The subjective factors (SIS 9-15) are as follows: SIS 9, alleged purpose of the suicide attempt; SIS 10, expectations of fatality; SIS 11, conception of a method’s lethality; SIS 12, seriousness of the suicide attempt; SIS 13, attitude toward living or dying; SIS 14, conception of medical rescuability; and SIS 15, degree of premeditation.

    Utilization and Reprocessing of Confounders in the ER

    We also used clinical data reported from the ER as confounding variables in prediction modeling. In total, 14 confounders were considered, including sex, age, marital status, religion, monthly income, living status, educational level, urbanicity, ER visit date, ER visit on a weekend, ER visit time, admission route, admission transportation, and discharge date. All of the confounders were collected as numerical values, with coded indices or quantitative values as follows: sex (1=male, 2=female), marital status (1=single, 2=married, 3=living together, 4=separated, 5=divorced, 6=widowed), religion (1=Christian, 2=Buddhist, 3=Catholic, 4=Atheist, 5=other), living status (1=living with family, 2=living with somebody, 3=group facilities, 4=living alone), educational level (1=none, 2=elementary school, 3=middle school, 4=high school, 5=undergraduate or higher), ER visit on a weekend (1=yes, 2=no), admission route, admission transportation, and monthly income (self-reported in Korean currency). To represent the date records numerically (ie, year-month-day, ER visit date, and discharge date), we transformed the original date into a decimal year value (eg, 2013-6-30=2013.492). The detailed equation for the numerical transformation of the dates is presented in the supplementary source code [21].

    Relationships Among Suicidal Intent Items

    For each pair of 15 suicidal intent items for each individual, we generated three relationship signatures comprising the interaction terms (I), harmonized average (H), and geometric angle differences using the tangent function (T). The definitions of the three relationships between the ith and jth element of SIS in the pth individual (R(I, H, T)) are represented by the following equations:

    where Sip indicates the ith SIS element in the pth individual, and Ii,jp determines the level of interaction between the ith and jth SIS elements. To represent the overall intensity in a sensitive manner, we utilized the harmonic mean of a pair of elements (Hi,jp). According to the differences in the sequential combination of a pair of elements, such as [Sip=2 > Sjp=3] and [Sip=3< Sjp=2], Ti,jp presents a single scalar value for the paired elements.

    Data Analysis

    The chi-square test and Student t test were performed to compare variables for suicide attempts with lethal and nonlethal medical outcomes.

    Missing Data Imputation

    The k-nearest neighbors algorithm in the R package bnstruct [22] was utilized to impute any missing values (the proportion of missing values in our data was 2.7%).

    Network Analysis

    Network model analysis was performed to build a relational model of lethal and nonlethal suicide attempts. Using the graphical lasso (GLASSO) method, we investigated the weighted correlations between the assessed SIS elements according to attempt fatalities [23]. The network comprised nodes representing the suicidal intent elements, and the edges depicted the relationships among nodes as the medical outcomes of those who had lethal and nonlethal suicide attempts (Figure 1). The statistical significance levels of the GLASSO results were determined using the random 10,000-permutation method (P<.05). R [22] and Cytoscape [24] were employed for data analysis and visualization of the results, respectively. The source code was deposited in the GitHub database [21].

    Machine Learning

    Machine learning techniques comprising random forest and linear regression were used. To evaluate the contributions of the relationship scores, we compared the predictive performances of models with or without the relationship features. We utilized TensorFlow [25] to develop our deep learning model called E-GONet. In addition, the feature importance map for the input data of convolutional neural network (CNN) models was generated by DeepExplain with the “Gradient*Input” method [26]. To obtain the feature map, the feature importance scores of all nonlethal cases and all lethal cases were averaged in each fold of 10-fold cross validation sets, and then, the average scores in each fold were averaged into final feature importance scores for nonlethal and lethal cases, respectively.


    Characteristics of Lethal and Nonlethal Outcomes of Suicide Attempts

    Among 1112 individuals who attempted suicide, 190 (17.1%) had suicide attempts categorized as lethal medical outcomes (Table 1). According to the C-SSRS–based fatality of attempt outcomes, we classified the individuals as those who had lethal attempts (n=190, C-SSRS severity 3) and those who had nonlethal attempts (n=922). More male individuals had lethal suicide attempts than nonlethal suicide attempts (107/190, 56.3% vs 357/922, 38.7%). The mean age of those who had lethal suicide attempts was higher than that of those who had nonlethal suicide attempts (47.3 years vs 42.3 years).

    The mean total SIS score was higher for those who had lethal suicide attempts than those who had nonlethal suicide attempts (30.23, SD 6.19 vs 25.06, SD 5.61). The total SIS score is the sum of the 15 graded elements of the SIS, such as the scale of isolation from 1 to 3 (complete isolation).

    Table 1. Demographic characteristics.
    FeaturesTotala (n=1112)Lethal attemptsa,b (n=190)Nonlethal attemptsa,b (n=922)P value
    Sex


    		<.001c

    		Female647 (58.18)83 (43.68)564 (61.17)

    		Male464 (41.73)107 (56.32)357 (38.72)

    		Unknown1 (0.09)0 (0.0)1 (0.11)
    Mean age, years43.17 (18.2)47.31 (18.41)42.31 (18.06)<.001d
    History of suicide attempts


    		.88c

    		No attempt or unknown763 (68.62)129 (67.89)634 (68.76)

    		Previous history349 (31.38)61 (32.11)288 (31.24)
    C-SSRSe fatality rating for the outcome of an attempt

    		1: None or minor physical damage177 (15.92)N/Af177

    		2: Minor physical damage387 (34.80)N/A387

    		3: Moderate physical damage358 (32.19)N/A358

    		4: Severe physical damage152 (13.67)152 (80.00)N/A

    		5: Very severe physical damage33 (2.97)33 (17.37)N/A

    		6: Death5 (0.45)5 (2.63)N/A
    Mean of the SISg sum26.74 (5.91)30.23 (6.19)25.06 (5.61)<.001d

    aData are presented as n (%), n, or mean (SD).

    bFatality scale of the Columbia Suicide Severity Rating Scale (3 for a lethal suicide attempt and <3 for a nonlethal suicide attempt).

    cChi-square test between those who had lethal attempts and those who had nonlethal attempts.

    dt test between those who had lethal attempts and those who had nonlethal attempts.

    eC-SSRS: Columbia Suicide Severity Rating Scale.

    fN/A: not applicable.

    gSIS: Beck Suicide Intent Scale.

    Network Model Based on Suicidal Intent Elements

    Using GLASSO, we determined the weighted correlations among the assessed SIS elements according to suicide attempt fatalities [23]. We constructed two networks comprising nodes representing the SIS elements (eg, degree of isolation) and edges depicting the relationships among nodes, which indicated the distinct relationships between the elements of suicide in those who had lethal and those who had nonlethal suicide attempts (Figure 1). The statistical significance of the correlations assessed among the SIS elements using GLASSO were determined based on random distributions of the SIS elements (P<.05 for random distributions). Finally, we represented the distinct relationships between the suicidal intents of those who attempted suicide (lethal and nonlethal cases).

    Fifteen nodes for suicidal intents were linked via 17 edges for lethal suicide attempts (n=190) (Figure 2A). Among the 922 individuals who had nonlethal suicide attempts, there were 16 relationships (ie, edges) among 14 suicidal intent elements (ie, nodes) in the nonlethal suicide attempts (Figure 2B). The edges between nodes represent the positive or negative relationships between nodes based on the GLASSO results (P<.05 for edges based on a random distribution). Among individuals who had lethal attempts, the fatal outcomes of suicide attempts were more tightly linked (ie, associated) with the suicidal intents compared with those who had nonlethal attempts, and the nodes for the SIS elements were loosely connected or separated in those who had nonlethal attempts (Figure 2A and B). Thus, the close connectedness of the suicidal intent elements, including the concept of lethality of the method, was stronger among those who had lethal attempts.

    The topological properties of the network, such as the central node that had the largest number of relationships with other nodes and the average of the shortest paths (ie, degree of centrality), were employed to determine the central suicidal intent elements for the lethal and nonlethal attempts (Figure 2C-F). Among the lethal suicide attempts, the fatality of suicide attempt (“Fatality” node) and the conception of a suicide method’s lethality (“ConMeth” node) were strongly associated with other suicidal intent elements (Figure 2E). In Figure 2C, the y-axis denotes the average shortest path, which was a bottleneck and a central node, and the “ConMeth” node was ranked highly among those who had lethal attempts (Figure 2C). However, the alleged purpose of suicide attempt (“APurpo” node), including “to manipulate the environment,” was a crucial intent element in the minds of those who had nonlethal attempts (Figure 2D and F). Moreover, among those who had nonlethal attempts, suicide method-related features (eg, the conception of a method’s lethality and the expectation of fatality), which are closely linked to lethality among those who had lethal attempts, were completely disconnected from the other suicidal intent nodes (Figure 2D and F).

    Thus, we elucidated the relationships between the suicidal intent elements in those who had lethal and those who had nonlethal suicide attempts. The conception of a method’s lethality (“ConMeth” node) was a central suicidal intent element, which was clearly related to lethal suicide attempts, and it was connected with the initiation of attempts, such as the nodes for expectation of fatality (“ExFatal” node) and seriousness of attempt (“SeriAtt” node).

    ‎
    Figure 2. Network structures obtained for lethal and nonlethal outcomes. (A, B) Networks obtained for lethal (A; n=190, C-SSRS fatality ≥3) and nonlethal (B; n=922, C-SSRS fatality <3) cases. Each node represents the SIS element (green circles) and assessed fatality of the attempt using the C-SSRS score (red circles). The linked edges indicate strong relationships between nodes based on the weighted correlations obtained by graphical lasso (GLASSO) (P<.05). The red edges represent positive relationships, and the blue edges represent negative relationships. The cyan circular nodes indicate SIS elements, and the red circular nodes indicate the C-SSRS fatality scores for suicide attempts. (C-F) Bar charts showing the topological properties of the networks obtained for lethal (C, E) and nonlethal cases (D, F). The SIS elements were as follows: isolation (Iso), time intervention feasibility (Tinter), active or passive precautions against discovery intervention (ArecInter), acting to get help (ActHelp), final acts in anticipation of death (FinActD), active preparation for attempt (ActPrep), suicide note (SNote), overt communication of suicidal intent (OvertSI), alleged purpose of attempt (Apurpo), expectation of fatality (ExFatal), conception of method lethality (ConMeth), seriousness of attempt (SeriAtt), attitude toward living or dying (AttiLiv), conception of medical rescuability (ConResc), and degree of premeditation impulsiveness (Dimpuls). C-SSRS: Columbia Suicide Severity Rating Scale; SIS: Beck Suicide Intent Scale.
    View this figure

    Predictive Model for Medical Lethality of Suicide Attempts Based on Deep Learning: E-GONet

    Owing to the structural differences in the networks of antecedent behaviors according to the lethal or nonlethal outcomes of the suicide attempts, we generated three relationship signatures for each pair of SIS elements comprising the interaction terms (I), harmonized average (H), and geometric angle differences using the tangent function (T). In addition to the 15 SIS elements and 14 types of clinically reported data collected by the ER, including age, admission date, and living status, three relationship signatures were prepared for all possible SIS combinations for each individual. We represented the pairs of SIS elements as specific numeric values, and 315 relationship features were obtained among the 105 combinations of SIS elements for an individual. We built E-GONet based on a CNN, which is a subclass of deep learning. The overall structure of E-GONet comprises input and output layers, as well as multiple hidden layers (convolutional layers, pooling layers, and fully connected layers). The schematic structure of each layer is shown in Figure 3A. Multimedia Appendix 1 and Multimedia Appendix 2 describe the detailed structures of the E-GONet model.

    ‎
    Figure 3. Construction of E-GONet and performance evaluation. (A) Structure of E-GONet based on a convolutional neural network model. The input data format was 18 × 20 (row × column), which comprised SIS or other features and SIS relationship features. The SIS or other features comprised 29 features (15 SIS features and 14 observations collected by emergency rooms) and seven blanks with all zero values (gray). The SIS relationship features comprised 315 features (105 relationships × three types) and nine blanks. E-GONet has three convolutional layers and two fully connected layers. TensorFlow 1.8.0 was used for the implementation. (B) Mean performance based on 10-fold cross-validation using all of the implemented features in A. The red bar shows the average performance of E-GONet with the 10-fold cross-validation set. (C) Mean performance using the data set without SIS relationship features. AUPRG: area under the precision-recall gain curve; F1: weighted-F1 score; NPV: negative-predictive value; PPV: positive-predictive value; SIS: Beck Suicide Intent Scale.
    View this figure

    Evaluation of E-GONet for Identifying the Medical Lethality of Suicide

    Figure 3B shows the results obtained from the performance evaluation. We evaluated the predictive performance of the E-GONet model by 10-fold cross-validation. We used the same dataset for the performance comparison with E-GONet and to establish two prediction models with linear regression and random forest (ie, an aggregated decision tree model) [27] methods.

    E-GONet performed better than the linear regression and random forest methods (E-GONet increased the F1 score up to 3.4%, and the mean increase in the F1 score was 2.1%; E-GONet also increased the area under the precision-recall gain curve [AUPRG] up to 32.1%, and the mean increase in the AUPRG was 18.1%) [28]. Besides, the positive-predictive value (PPV; precision) comprising the rate of correctly identifying lethal attempts was highest with the E-GONet predictions (0.59). As generally noted in the clinical field, our dataset was relatively imbalanced (lethal 190, nonlethal 922). In analysis involving imbalanced data, sensitivity, PPV, F1, and AUPRG have been used for performance evaluation instead of specificity, negative-predictive value, and accuracy.

    The analysis of the contribution of learning features (Multimedia Appendix 3 and Multimedia Appendix 4) showed that most of the contributions of confounding variables (such as level of education) were negligible for the predictive performance of E-GONet. However, the relationship signatures between the SIS element pairs contributed greatly to the superior performance of the E-GONet model (the values of R(I, H, T)). As depicted in Multimedia Appendix 3 and Multimedia Appendix 4, the saliency heatmaps of our model highlighted the contribution of SIS relationship features. The first two rows of the feature importance matrixes were the confounders (age, sex, income levels, etc) and SIS elements (SIS 1-15). Out of those features, only age and income level contributed to E-GONet training. On the other hand, as presented in the figures, the developed relationship features of SIS elements were more important for CNN model training. Interestingly, out of all relationships among SIS elements, the relationship with SIS element 11 (conception of a suicide method’s lethality) was the biggest contributor to predictions. This is highly consistent with the network modeling of SIS elements in Figure 2.

    Moreover, Figure 3C shows the evaluations of the performance of the models established without the SIS-based relationship features. The predictive models based on linear regression and random forest exhibited similar or lower performance after introducing the SIS-based relationship features, because these classical methods could not patternize the relationship features of suicide elements. However, E-GONet trained and improved performance via the vectorized relationships in high-dimensional spaces. As a result, the relationships between SIS elements increased the performance of the E-GONet model by 60% in precision, 6% in F1, and 8% in AUPRG (precision [without relationship/with relationship]=0.0/0.59, F1=0.75/0.81, and AUPRG=0.65/0.73). The full spectrum of the AUPRG displayed the training and fitting process of our deep learning approach in a very detailed manner (Multimedia Appendix 5). In Multimedia Appendix 6, the standard deviations of AUPRGs are presented via 100 trials of 10-fold cross-validation settings.

    Therefore, identifying the lethality of attempt outcomes is feasible with deep learning through the major contributions of the relationships among SIS elements (ie, mutual interactions of antecedent behaviors). To allow the use of our method in clinics, we have made all of the source codes for the analytics available via the internet, including the network-based analytics, E-GONet model, and data preprocessing methods [21].


    Using network analysis, we elucidated the relationships among antecedent behaviors prior to suicide attempts, where we identified the unique patterns associated with both lethal and nonlethal medical outcomes of suicide attempts. These findings allowed us to interpret the behaviors before lethal suicide attempts, thereby helping us to systematically investigate the interactions and connections among the behaviors that result in lethal suicide attempts. In particular, suicide attempts with lethal medical outcomes were associated with clear concepts regarding the likely fatality of the methods applied. In addition, behaviors, such as isolation at the time of suicide attempts, were strongly associated with the expected intervention time and the possibility of being discovered by other people. Among nonlethal suicide attempts, the suggested aim of suicide was a central factor among suicidal intent elements. Thus, lethal suicide attempts involved clear notions regarding the success of suicide, whereas nonlethal attempts were focused on the achievement of suicide attempts per se. In addition, prediction based on deep learning performed better after introducing the relationship signatures among the suicidal intent elements (60% increase in precision, 6% increase in the F1 score, and 8% increase in the AUPRG). Based on the analysis of feature contributions, we conclude that training of the relationships among SIS elements, especially isolation and the conception of a method’s lethality, strongly ameliorated the deep learning performance. To the best of our knowledge, this is the first study to successfully discern the differences in mutual interactions among antecedent communicative behaviors prior to suicide attempts by those who had lethal and those who had nonlethal attempts, in which our novel method employed relational signatures to facilitate deep learning–based predictions.

    In this study, we found that suicide attempts in individuals who had information about suicide methods and who anticipated fatality before attempting suicide had more lethal consequences. Our previous study showed that suicide methods are highly associated with subsequent suicide-related death [29]. Based on these results, we can infer that possessing information about suicide methods and their severity will affect suicide attempts and the consequent lethal outcome. Information about suicide methods can be found easily via the internet, and previous studies have shown that online searches for suicide-related terms are positively associated with intentional self-injury and death due to suicide [30,31]. In addition, we need to consider that suicide methods are subject to cultural differences. For example, suicide methods employed in the United States are predominantly related to firearms and the suicide rate is related to the gun possession rate by state [32]. By contrast, gun usage is very rare in South Korea because of the legal regulations related to gun possession [33]. However, the use of pesticides is fairly prevalent in suicide attempts in Korea, especially in rural areas and among elderly individuals who attempt suicide [17]. According to our results, we believe that restricting the accessibility of information regarding suicide methods is essential for suicide prevention, and cultural differences should be considered.

    In previous studies, a machine learning algorithm trained with the longitudinal electronic health records of patients reliably predicted suicidal behavior [34] and actual suicide among US Army soldiers [35]. Linguistic-driven models that use the text in clinical notes have also been explored, but they lack sufficient accuracy (approximately 65%) [36]. In the future, machine learning based on medical big data may become a ubiquitous component of clinical research and practice, which is a prospect that some find uncomfortable [37]. This study was based on three components comprising psychiatric physicians, data scientists, and a sophisticated computational infrastructure (KAT GPU Cluster System, Intel Xeon Ivy Bridge, 2.50 GHz 10 Cores; NVIDIA Tesla V100). However, the contributions of relationship features to the precise fatality predictions demonstrate that insights from physicians, including our hypothesis (ie, interactions among SIS elements were useful), as well as communication with the algorithm developer, are essential for innovative digital health development and precision medicine.

    We have developed new approaches to investigate the characteristics of suicide attempts; however, this study had several limitations. First, the study sample did not represent the whole population of individuals who attempt suicide, as the 17 medical centers were located in specific urban areas of South Korea. The sample only included individuals who attempted suicide and came to the emergency centers [17]. In addition, the characteristics of suicide attempts differ among cultures. However, despite the limitations of the sample, the 1359 individuals who had suicide attempts comprised a large number of those who were assessed by a clinician shortly after their suicide attempts. The 17 medical centers were selected based on their enrollment in the National Emergency Department Information System, which is a government-managed nationwide registration system [38]. Lastly, E-GONet may have additional costs for learning and operating the deep learning model, as a deep learning model would require a more specialized facility with systems like a GPU system. Thus, cost-effectiveness and streamlined operation are the next milestones for deep learning–based approaches. For example, the world’s best artificial intelligence model AlphaGo has a cost of US $35 million. This is much higher than the cost of a single human Go player per game.

    As is usually noted in the clinical field, our data were relatively imbalanced (lethal, n=190; nonlethal, n=922). In order to appropriately analyze the data, we tried to use data resampling approaches. We applied an over-sampling method, an under-sampling method, and the synthetic minority over-sampling technique (SMOTE), but these methods did not improve the results. Therefore, we did not apply resampling approaches to our analysis. In addition, since resampling methods could not improve the results, it is expected that applying cost-sensitive loss functions will also not change the results.

    In this study, we performed binary classification based on a lethality threshold (lethal >3, nonlethal ≤3). If we perform more fine-grained classification (ie, predict the exact C-SSRS grade), we could obtain more information for tailored care in clinics. However, as depicted in Table 1, the outcome of suicide attempts (C-SSRS grade) can be classified into six levels. Among the six levels, levels 5 and 6 involved very limited numbers of individuals who attempted suicide. Thus, we can only build a regression model for minor physical damage (ie, C-SSRS grades 1, 2, and 3) and severe damage (C-SSRS grade 4). However, because the number of individuals in C-SSRS grade 4 is limited compared with minor damage cases, the regression model for severe damage may not be well developed. Therefore, more fine-grained classification is not appropriate in this study. However, binary classification can provide clinically meaningful information. Regardless of our study design, the fine-grained identification of suicide attempts can be a guideline for further studies.

    Two conclusions can be drawn regarding the originality of this study. First, effective management strategies can be provided for the care of individuals who attempt suicide, as individuals with lethal outcomes had expectations regarding the fatality of their suicide attempts and they made great preparations before their attempts so that they would not be found. Second, the enhanced performance of deep learning prediction shows that preprocessing the relationships in patient data using nonlinear transformation (ie, the interaction terms between SIS elements) can help the machine learning process understand the information embedded in clinical practice and employ it to make effective inferences.

    The findings of this study may help public health officials and clinicians to identify the profiles of individuals at high risk of recurrent suicide attempts and may facilitate the development of efficient and effective suicide prevention programs.

    Acknowledgments

    This study was supported by the National Institute of General Medical Sciences of the National Institutes of Health (award number R01GM079719) and by the Korea Institute of Science and Technology Information (KISTI) (K-20-L02-C10-S01, K-17-L03-C02-S01, and P-18-SI-CT01-S01). The computational analysis was supported by the National Supercomputing Center, including resources and technology (K-18-L12-C08-S01, KAT GPU cluster system). YMA was supported by the Korean Ministry of Health (Korea National Suicide Survey [KNSS]).

    Authors' Contributions

    YMA and HP are both corresponding authors. BK and YK contributed equally to this manuscript. HP designed and wrote the paper; HP prepared the major figure, wrote the entire manuscript, designed the main algorithm, and analyzed the data; BK and YMA contributed to writing the manuscript and provided all of the data employed; YK conducted the deep learning process (E-GONet), feature contribution analysis, and background analysis; HKP and SJR contributed useful comments to writing the manuscript; YSK and BLL supported the project; and YMA and HP organized the overall project. All authors discussed the results and implications and commented on the manuscript at all stages.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Structure of E-GONet, which is a convolutional neural network model.

    PNG File , 202 KB
    ‎
    Multimedia Appendix 2

    Structure of E-GONet (without relationship features).

    PNG File , 138 KB
    ‎
    Multimedia Appendix 3

    Importance of the utilized features for the training of E-GONet.

    PNG File , 160 KB
    ‎
    Multimedia Appendix 4

    Detailed contribution of the Beck Suicide Intent Scale relationship features.

    PNG File , 153 KB
    ‎
    Multimedia Appendix 5

    Precision-recall gain curves for random forest, linear regression, and E-GONet (with relationship features).

    PNG File , 149 KB
    ‎
    Multimedia Appendix 6

    Performance of the area under the precision-recall gain curve.

    PNG File , 124 KB
    1. Caine ED. Suicide and Attempted Suicide in the United States During the 21st Century. JAMA Psychiatry 2017 Nov 01;74(11):1087-1088. [CrossRef] [Medline]
    2. WISQARS Leading Causes of Death Reports. Centers for Disease Control and Prevention.   URL: https://webappa.cdc.gov/sasweb/ncipc/leadcause.html [accessed 2018-12-28]
    3. Lim D, Ha M, Song I. Trends in the leading causes of death in Korea, 1983-2012. J Korean Med Sci 2014 Dec;29(12):1597-1603 [FREE Full text] [CrossRef] [Medline]
    4. O'Connor RC, Nock MK. The psychology of suicidal behaviour. Lancet Psychiatry 2014 Jun;1(1):73-85. [CrossRef] [Medline]
    5. James K, Stewart D, Bowers L. Self-harm and attempted suicide within inpatient psychiatric services: a review of the literature. Int J Ment Health Nurs 2012 Aug;21(4):301-309. [CrossRef] [Medline]
    6. Cavanagh JT, Carson AJ, Sharpe M, Lawrie SM. Psychological autopsy studies of suicide: a systematic review. Psychol Med 2003 Apr;33(3):395-405. [CrossRef] [Medline]
    7. Mann JJ, Arango VA, Avenevoli S, Brent DA, Champagne FA, Clayton P, et al. Candidate endophenotypes for genetic studies of suicidal behavior. Biol Psychiatry 2009 Apr 01;65(7):556-563 [FREE Full text] [CrossRef] [Medline]
    8. Beck AT, Beck R, Kovacs M. Classification of suicidal behaviors: I. Quantifying intent and medical lethality. Am J Psychiatry 1975 Mar;132(3):285-287. [CrossRef] [Medline]
    9. Kumar CT, Mohan R, Ranjith G, Chandrasekaran R. Characteristics of high intent suicide attempters admitted to a general hospital. J Affect Disord 2006 Mar;91(1):77-81. [CrossRef] [Medline]
    10. Rosen DH. The serious suicide attempt: epidemiological and follow-up study of 886 patients. Am J Psychiatry 1970 Dec;127(6):764-770. [CrossRef] [Medline]
    11. Cramer AO, Waldorp LJ, van der Maas HL, Borsboom D. Comorbidity: a network perspective. Behav Brain Sci 2010 Jun;33(2-3):137-50; discussion 150. [CrossRef] [Medline]
    12. Borsboom D, Cramer AO. Network analysis: an integrative approach to the structure of psychopathology. Annu Rev Clin Psychol 2013;9:91-121. [CrossRef] [Medline]
    13. Boschloo L, van Borkulo CD, Rhemtulla M, Keyes KM, Borsboom D, Schoevers RA. The Network Structure of Symptoms of the Diagnostic and Statistical Manual of Mental Disorders. PLoS One 2015;10(9):e0137621 [FREE Full text] [CrossRef] [Medline]
    14. Bringmann LF, Vissers N, Wichers M, Geschwind N, Kuppens P, Peeters F, et al. A network approach to psychopathology: new insights into clinical longitudinal data. PLoS One 2013;8(4):e60188 [FREE Full text] [CrossRef] [Medline]
    15. van Borkulo C, Boschloo L, Borsboom D, Penninx BW, Waldorp LJ, Schoevers RA. Association of Symptom Network Structure With the Course of [corrected] Depression. JAMA Psychiatry 2015 Dec;72(12):1219-1226. [CrossRef] [Medline]
    16. Fitzpatrick KK, Darcy A, Vierhile M. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Ment Health 2017 Jun 06;4(2):e19 [FREE Full text] [CrossRef] [Medline]
    17. Kim B, Ahn J, Cha B, Chung Y, Ha TH, Hong Jeong S, et al. Characteristics of methods of suicide attempts in Korea: Korea National Suicide Survey (KNSS). J Affect Disord 2015 Dec 01;188:218-225. [CrossRef] [Medline]
    18. Pai D, Woo J, Son MH, Lee C. The Reliability and Validity of the Korean Version of Columbia-Suicide Severity Rating Scale in Alcohol Dependent Patients. J Korean Neuropsychiatr Assoc 2015;54(2):222. [CrossRef]
    19. Serrani Azcurra D. Psychometric validation of the Columbia-Suicide Severity rating scale in Spanish-speaking adolescents. Colomb Med (Cali) 2017 Dec 30;48(4):174-182 [FREE Full text] [CrossRef] [Medline]
    20. Beck RW, Morris JB, Beck AT. Cross-validation of the Suicidal Intent Scale. Psychol Rep 1974 Apr;34(2):445-446. [CrossRef] [Medline]
    21. Kim Y, Paik H. The suicide network source code. GitHub database.   URL: https://github.com/hypaik/SuicideNetwork [accessed 2019-04-26]
    22. Sambo F, Franzin A. bnstruct: Bayesian Network Structure Learning from Data with Missing Values. CRAN.   URL: https://cran.r-project.org/web/packages/bnstruct/index.html [accessed 2018-08-31]
    23. Bryant RA, Creamer M, O'Donnell M, Forbes D, McFarlane AC, Silove D, et al. Acute and Chronic Posttraumatic Stress Symptoms in the Emergence of Posttraumatic Stress Disorder: A Network Analysis. JAMA Psychiatry 2017 Feb 01;74(2):135-142. [CrossRef] [Medline]
    24. Shannon P, Markiel A, Ozier O, Baliga NS, Wang JT, Ramage D, et al. Cytoscape: a software environment for integrated models of biomolecular interaction networks. Genome Res 2003 Nov;13(11):2498-2504 [FREE Full text] [CrossRef] [Medline]
    25. TensorFlow.   URL: http://www.tensorflow.org [accessed 2020-06-16]
    26. Ancona M, Ceolini E, Öztireli C, Gross M. Towards better understanding of gradient-based attribution methods for deep neural networks. 2018 Presented at: 6th International Conference on Learning Representations ICLR; 2018; Vancouver. [CrossRef]
    27. Pflueger MO, Franke I, Graf M, Hachtel H. Predicting general criminal recidivism in mentally disordered offenders using a random forest approach. BMC Psychiatry 2015 Mar 29;15:62 [FREE Full text] [CrossRef] [Medline]
    28. Flach PA, Kull M. Precision-Recall-Gain curves: PR analysis done right. Adv Neural Inf Process Syst 2015 [FREE Full text]
    29. Kim B, Lee J, Kim E, Kim SH, Ha K, Kim YS, et al. Sex difference in risk period for completed suicide following prior attempts: Korea National Suicide Survey (KNSS). J Affect Disord 2018 Feb;227:861-868. [CrossRef] [Medline]
    30. McCarthy MJ. Internet monitoring of suicide risk in the population. J Affect Disord 2010 May;122(3):277-279 [FREE Full text] [CrossRef] [Medline]
    31. Yang AC, Tsai SJ, Huang NE, Peng CK. Association of Internet search trends with suicide death in Taipei City, Taiwan, 2004-2009. J Affect Disord 2011 Jul;132(1-2):179-184. [CrossRef] [Medline]
    32. Karp A. Completing the Count: Civilian firearms - Annexe online. In: Small Arms Survey 2007: Guns and the City; Chapter 2. Cambridge: Cambridge University Press; 2007.
    33. Newton GD, Franklin EZ. Firearm Licensing: Permissive v Restrictive. In: Firearms & Violence in American Life: A staff report submitted to the National Commission on the Causes and Prevention of Violence. Washington, DC: US Government Printing Office; 1969.
    34. Barak-Corren Y, Castro VM, Javitt S, Hoffnagle AG, Dai Y, Perlis RH, et al. Predicting Suicidal Behavior From Longitudinal Electronic Health Records. Am J Psychiatry 2017 Feb 01;174(2):154-162. [CrossRef] [Medline]
    35. Kessler RC, Warner CH, Ivany C, Petukhova MV, Rose S, Bromet EJ, Army STARRS Collaborators. Predicting suicides after psychiatric hospitalization in US Army soldiers: the Army Study To Assess Risk and rEsilience in Servicemembers (Army STARRS). JAMA Psychiatry 2015 Jan;72(1):49-57 [FREE Full text] [CrossRef] [Medline]
    36. Poulin C, Shiner B, Thompson P, Vepstas L, Young-Xu Y, Goertzel B, et al. Predicting the risk of suicide by analyzing the text of clinical notes. PLoS One 2014;9(1):e85733 [FREE Full text] [CrossRef] [Medline]
    37. Adkins DE. Machine Learning and Electronic Health Records: A Paradigm Shift. Am J Psychiatry 2017 Feb 01;174(2):93-94 [FREE Full text] [CrossRef] [Medline]
    38. Cha W, Ahn K, Shin SD, Park J, Cho J. Emergency Department Crowding Disparity: a Nationwide Cross-Sectional Study. J Korean Med Sci 2016 Aug;31(8):1331-1336 [FREE Full text] [CrossRef] [Medline]

    ‎
    AUPRG: area under the precision-recall gain curve
    CNN: convolutional neural network
    C-SSRS: Columbia Suicide Severity Rating Scale
    ER: emergency room
    GLASSO: graphical lasso
    PPV: positive-predictive value
    SIS: Beck Suicide Intent Scale

    Edited by G Eysenbach; submitted 28.04.19; peer-reviewed by G Lim, S Kim; comments to author 12.08.19; revised version received 08.01.20; accepted 23.03.20; published 09.07.20

    Copyright

    ©Bora Kim, Younghoon Kim, C Hyung Keun Park, Sang Jin Rhee, Young Shin Kim, Bennett L Leventhal, Yong Min Ahn, Hyojung Paik. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 09.07.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.