Analysis and Prediction of Relapse Risk Factors in Community-Dwelling Individuals With Schizophrenia: Retrospective Cohort Study

Introduction

Schizophrenia is a major global public health concern characterized by multidimensional impairments in perception, cognition, and emotion [1]. According to the Global Burden of Disease (GBD) Study 2019, an estimated 23.6 million people worldwide are affected by schizophrenia [2], with an increasing incidence over time [3]. Its high relapse rate severely compromises functional recovery and poses substantial challenges to long-term disease management, particularly within community-based mental health systems [4-6], further threatening the safety and well-being of families and communities [7].

The early identification and prediction of relapse have long been priorities in schizophrenia research [8-10]. Previous studies have reported that factors such as medication adherence, substance misuse, insight [11], exposure to stressful life events [12], social support, employment status [13], sleep quality [8], and adverse drug reactions [14] are closely associated with relapse [15]. Although electronic health records (EHRs) hold great potential for relapse or rehospitalization prediction, several limitations persist [16]. Relapse is widely recognized as the result of complex interactions among biological, psychological, and social factors, yet systematic investigations within community-based rehabilitation settings remain scarce. In practice, most patients with schizophrenia rely primarily on long-term community follow-up for disease management. However, existing studies often rely on data from single medical institutions with limited sample sizes, restricting their ability to reflect the real-world disease trajectory. Additionally, data collection methods vary across settings, and some variables depend on subjective assessments, which may weaken model generalizability and external validity.

While machine learning techniques offer advantages over traditional statistical methods in handling high-dimensional, complex real-world data and in distinguishing relapse from nonrelapse cases [17], existing models still face limitations in robustness and scalability [13,18,19]. In community mental health systems, predictive models must not only achieve high accuracy but also demonstrate interpretability and operational feasibility, enabling frontline practitioners to derive actionable intervention insights.

To address these gaps, this study developed an information management system (IMS) dataset encompassing community rehabilitation institutions across Shanghai and including 39,818 individuals with schizophrenia. The dataset integrates clinical diagnostic information, demographic characteristics, and community follow-up records, providing a comprehensive representation of relapse-related risk characteristics in real-world community mental health management settings. Furthermore, explainable machine learning techniques were used to improve predictive performance while elucidating the relative contributions of different risk factors to model predictions. The aim of this study was to develop a decision-support tool that can be integrated into community mental health service workflows to support risk identification and timely intervention, thereby providing empirical evidence for the monitoring of hospitalization-related severe relapse risk among individuals with schizophrenia in community settings.

Methods

Study Design and Setting

This study was based on the IMS for community-based rehabilitation of patients with schizophrenia, developed and maintained by the Shanghai Mental Health Center; the system continuously monitors and records patients’ rehabilitation data across 16 districts. Patients experiencing acute or severe psychotic episodes were admitted to psychiatric hospitals for inpatient treatment, while those in clinically stable phases were transferred to home-based and community-based rehabilitation programs. Within these community rehabilitation centers, psychiatrists at outpatient clinics were responsible for recording and uploading follow-up information to the IMS. Additionally, family members or other informal caregivers were invited to participate in the evaluation process, providing supplementary observations of patients’ behavior and daily functioning, thereby contributing to the establishment of a unified and comprehensive IMS database.

Sampling Strategy and Participants

A stratified cluster sampling strategy was adopted to select participants for this study. First, based on the level of urbanization, the 16 administrative districts of Shanghai were categorized into 3 tiers: central urban districts, semicentral suburban districts, and suburban districts, which served as the sampling framework. Following the principle of cluster sampling, Huangpu District and Yangpu District were selected from the central urban tier, Pudong New Area from the semicentral suburban tier, and Baoshan, Minhang, and Jiading Districts from the suburban tier. All individuals with schizophrenia who had follow-up records in the Shanghai Mental Health Information System between January 2022 and December 2023 within the selected districts were included in the initial sample (n=44,804). Participants were then screened according to predefined inclusion and exclusion criteria.

The inclusion criteria were as follows: (1) officially registered in the Shanghai Mental Health IMS; (2) aged 18 years or older; and (3) diagnosed with schizophrenia according to the International Classification of Diseases, Tenth Revision (ICD-10) criteria. The exclusion criteria were as follows: deceased individuals, those who had migrated outside Shanghai during the data collection period, long-term inpatients, and individuals younger than 18 years. After applying these criteria, a total of 39,818 patients were retained for subsequent modeling and analysis.

During feature selection, personal identifiers (eg, patient name, hospital name, and rehabilitation site), time-dependent variables irrelevant to the prediction task (eg, follow-up date and record creation time), and potential data leakage variables (eg, residential address and name of attending psychiatrist) were excluded. Each patient record contained structured information, including demographic characteristics, clinician-rated diagnostic data, social interaction indicators, and treatment-related management details. Additionally, unstructured text fields documented behavioral observations, participation in rehabilitation programs, and individualized recommendations from psychiatric specialists.

In this study, relapse was operationally defined as a patient’s transfer from community-based rehabilitation to a psychiatric hospital for inpatient treatment due to acute exacerbation or deterioration of psychotic symptoms [20]. Hospitalizations related to physical comorbidities or nonpsychiatric causes were excluded. Only hospital admissions that psychiatrists determined to be acute psychotic relapse events were classified as relapse cases. If such a transfer occurred during the observation period, the patient was labeled as a positive (relapse) case; otherwise, the patient was classified as a negative (nonrelapse) case.

For relapse cases, data from the last follow-up prior to hospitalization were used as predictive variables for subsequent relapse outcomes. For nonrelapse cases, data from the most recent follow-up were selected as predictive variables, and the absence of relapse during the subsequent observation period was used as the outcome label. This approach ensured temporal precedence of predictors relative to outcomes, thereby preventing bias from potential data leakage.

Because the original IMS database could contain multiple follow-up records for the same patient, the analysis dataset was constructed at the patient level before model development. Each patient was restricted to 1 predefined index record. For patients who experienced hospitalization-related severe relapse, the index record was defined as the last community follow-up record before hospitalization due to acute exacerbation or deterioration of psychotic symptoms. For patients without hospitalization-related severe relapse during the observation period, the index record was defined as the last valid community follow-up record during the observation period. After this procedure, each patient contributed only one record to the final analysis dataset.

Feature Construction

The initial feature pool comprised 52 candidate variables, spanning five thematic domains: (1) sociodemographic characteristics; (2) clinical diagnosis and medical history; (3) medication management and follow-up; (4) rehabilitation participation; and (5) rehabilitation interventions and specialist recommendations. During data preprocessing, variables with more than 50% missing values were excluded [21]. Prior to feature selection, a variance thresholding approach was applied to remove variables with negligible predictive utility. Specifically, features with zero variance and near-zero variance (ie, nonzero occurrence rate below 0.1%) were eliminated. After preprocessing, 41 features were retained for model training (Table 1). These features were subsequently organized into 5 major domains (Multimedia Appendix 1), as summarized in Table 1.

Ethical Considerations

This study was based on the analysis of routinely collected community follow-up data, with all personally identifiable information removed prior to analysis. The Ethics Committee of the Shanghai Mental Health Center approved this study (institutional review board registration number: IORG0002202; Federalwide Assurance number: FWA00003065).

Statistical Analysis and Model Training

Statistical Analysis

Baseline statistical analyses were performed using SPSS Statistics software (version 26.0; IBM Corp). For categorical variables, chi-square tests were conducted to examine group differences in variable distributions across relapse statuses. For multiple-response categorical variables, multiple-response analyses were applied to calculate the frequency of each response option, followed by between-group comparisons. For continuous variables with skewed distributions or those violating the assumption of normality, nonparametric tests were used. Specifically, independent-samples t-tests or Mann-Whitney U tests were used for comparisons between 2 groups, whereas Kruskal-Wallis H tests were used for comparisons involving more than 2 groups. All statistical tests were 2-tailed, and a P value <.05 was considered statistically significant.

Model Training

In the present study, the original dataset exhibited severe class imbalance, with a ratio of approximately 10:1 between the majority (nonrelapse) and minority (relapse) classes. Such an extreme imbalance can cause conventional classifiers to exhibit strong bias toward the majority class, thereby achieving deceptively high overall accuracy but extremely poor recall for the minority class—resulting in suboptimal identification of relapse cases and limited generalizability.

A stratified random sampling strategy was applied to split the dataset into a training set (31,854/39,818, 80%) and an independent test set (7964/39,818, 20.0%), ensuring that the relapse-to-nonrelapse ratio remained consistent across both subsets. All feature selection and hyperparameter optimization procedures were strictly confined to the training set to prevent information leakage. Final model performance was evaluated exclusively on the held-out test set to ensure an unbiased assessment [22]. Class imbalance was addressed using cost-sensitive learning. For tree-based gradient boosting models (Light Gradient Boosting Machine [LightGBM] and extreme gradient boosting [XGBoost]), scale_pos_weight was applied. For random forest (RF), support vector machine (SVM), and logistic regression (LR), class_weight was used. For adaptive boosting (AdaBoost), sample weights (sample_weight) were assigned. Higher misclassification costs were assigned to the minority class (relapse), and all weighting parameters were jointly optimized with other hyperparameters through 5-fold stratified cross-validation and grid search. This strategy improves minority-class sensitivity without altering the true class distribution of the test set.

Missing-value handling was implemented according to model capabilities. LightGBM and XGBoost support native missing-value processing by learning an optimal routing direction for missing observations at each tree split; therefore, original missing values were retained without imputation. For the 4 models that do not support native missing values (RF, AdaBoost, SVM, and LR), median imputation was performed using statistics estimated exclusively from the training set. For SVM and LR, feature standardization was subsequently applied, with the scaler fitted only on the training set and then applied to the test set, thereby preventing any information leakage.

A total of 14 variables contained missing values, with missingness rates ranging from 0.6% to 17.2%. Under this framework, all samples retained their natural missing-data patterns and were handled through the native mechanisms of tree-based models whenever possible. Finally, model-derived feature importance was examined as part of the model interpretation process, and Shapley additive explanations (SHAP) analysis was subsequently used to quantify feature contributions to the final LightGBM model [23].

Exploratory Temporal Robustness Check

An exploratory temporal robustness check was conducted according to the calendar year of the retained patient-level index record. Using the same modeling pipeline as in the primary random 80/20 split analysis, the LightGBM model was trained on the 2022 index-record cohort and evaluated on the 2023 index-record cohort. Because the calendar year of the retained index record was partly influenced by the index-record selection rule, this analysis was prespecified as an exploratory robustness check rather than as a conventional independent temporal validation. For patients without hospitalization-related severe relapse, the retained index record was the last valid follow-up record during the observation period, whereas for relapse cases, the retained index record was the last follow-up before hospitalization. Therefore, the calendar-year split was not fully independent of outcome status.

Model Implementation and Optimization

The machine learning models were implemented in Python 3.10 using LightGBM (version 3.3.5). In this study, 6 widely used machine learning algorithms were evaluated, including LightGBM, RF, AdaBoost, SVM, LR, and XGBoost.

Hyperparameter optimization was performed using GridSearchCV with 5-fold stratified cross-validation, using area under the receiver operating characteristic curve (ROC-AUC) as the optimization metric. Class imbalance was addressed through cost-sensitive learning rather than resampling. Specifically, gradient boosting models (LightGBM and XGBoost) used the scale_pos_weight parameter, while RF, SVM, and LR used class_weight, and AdaBoost incorporated class-dependent sample weights (sample_weight). The hyperparameter search spaces and the corresponding optimal configurations for all models are provided in Multimedia Appendix 2.

SHAP Analysis

To enhance the clinical interpretability of the model, SHAP analysis was performed on the final trained LightGBM model to quantify the direction and magnitude of each feature’s contribution to the predicted outcome.

SHAP values are derived from Shapley values in cooperative game theory and provide both local explanations at the individual patient level and global feature importance at the cohort level.

In the SHAP summary plot, each point represents an individual patient. The x-axis indicates the SHAP value, which reflects the contribution of a given feature to the model output. Positive SHAP values indicate an increased predicted risk of relapse, whereas negative values indicate a lower predicted risk. Features are ranked according to their mean absolute SHAP values, with those at the top of the plot exerting a greater overall influence on model predictions.

Results

Baseline Characteristics of the Included Patients

The baseline characteristics of community-dwelling patients with schizophrenia showed that the median age was 60 (IQR 47‐69) years, and 53.2% (21,170/39,818) of the participants were female. Among the 39,818 individuals included in this study, 3717 (9.3%) experienced relapse, while 36,101 (90.7%) did not.

Between-group comparisons indicated that patients in the relapse group were significantly younger than those in the nonrelapse group (z=−20.78, P<.001). Regarding educational attainment, a significantly higher proportion of patients with a college education or above was observed in the relapse group (839/3717, 22.6%) compared with the nonrelapse group (χ²₂=98.73, P<.001). In terms of marital status, the proportion of unmarried patients was higher in the relapse group (2054/3717, 55.3%) than in the nonrelapse group (15,234/36,101, 42.2%; χ²₂=339.34, P<.001; Table 2). The detailed data provided in Multimedia Appendix 1.

Model Development and Performance

To systematically evaluate risk factors associated with relapse in schizophrenia and their relative importance, multiple machine learning models were constructed and compared, including LightGBM, AdaBoost, XGBoost, SVM, LR, and RF.

Given that nonrelapse cases were dominant in the dataset, relying solely on overall accuracy and weighted metrics (weighted recall, weighted precision, and weighted F₁-score) may be biased toward the majority class. Therefore, this study additionally focused on relapse-class-specific performance and macroaveraged metrics (macro recall, macro precision, and macro F₁-score) to more objectively evaluate model performance under class imbalance conditions.

Among the 6 models evaluated, the 2 gradient boosting models (LightGBM and XGBoost) achieved the best overall performance, both attaining ROC-AUC values of approximately 0.95 and substantially outperforming LR (ROC-AUC=0.85). These findings suggest that boosting-based models are more capable of capturing the complex nonlinear relationships and high-dimensional interactions present in community mental health EHRs (Figure 1). LightGBM demonstrated the best overall performance, achieving an ROC-AUC of 0.95, an area under the precision–recall curve (PR-AUC) of 0.79, a relapse-class recall of 0.77, a precision of 0.71, and an F₁-score of 0.74. The optimal hyperparameter configuration was learning_rate=0.1, max_depth=13, n_estimators=100, num_leaves=31, and scale_pos_weight=3.

‎

Notably, the models exhibited a clear sensitivity–precision trade-off in identifying relapse cases. AdaBoost and SVM achieved relatively high relapse-class recall values of 0.80 and 0.82, respectively, indicating greater sensitivity in detecting relapse cases. However, their precision values were comparatively low, at 0.49 and 0.56, respectively, suggesting a higher number of false-positive predictions. XGBoost showed overall performance highly comparable to that of LightGBM, with similar accuracy, macro F₁-score, relapse-class recall, relapse-class precision, relapse-class F₁-score, and ROC-AUC (Table 3). However, the PR-AUC of XGBoost was slightly lower than that of LightGBM. Considering that PR-AUC is particularly informative under class imbalance, and that LightGBM maintained a favorable balance between relapse-class recall and precision, LightGBM was retained as the primary model for subsequent interpretability and sensitivity analyses.

In addition to discrimination performance, model calibration was evaluated using calibration curves and Brier scores. The unweighted baseline model achieved a Brier score of 0.036, indicating good agreement between predicted probabilities and observed outcomes. The cost-sensitive LightGBM model yielded a slightly higher Brier score of 0.040. Although cost-sensitive learning improved relapse detection performance, a modest reduction in calibration was observed, reflecting the inherent trade-off between sensitivity and probability calibration (Multimedia Appendix 2).

Model Interpretability

The SHAP-based interpretability analysis identified the key predictors contributing to model predictions, ranked according to their relative importance (Figure 2). Among all features, medication persistence contributed the most to model prediction, accounting for 23.30% of the total feature importance, suggesting that medication-management information was an important predictive signal. The next most influential predictors were specialist recommendation: medication adherence training (19.74%), number of follow-ups (7.38%), and age (3.72%), indicating that these variables were closely related to the relapse risk patterns captured by the model. Other important predictors included educational attainment (3.58%), occupational functioning and productive work (3.11%), family supervision status (2.94%), and social functioning assessment (2.80%). The SHAP summary plot further showed the direction of associations between these features and model-predicted risk. Overall, refusal of medication, intermittent medication use, or poorer medication persistence during disease management, as well as a greater number of follow-ups, were generally associated with higher model-predicted risk. Although the number of follow-ups did not show a significant univariate between-group difference in the baseline comparison, its SHAP contribution reflects its conditional contribution to model prediction after accounting for other clinical, medication-management, and functional variables. In addition, younger age and higher educational attainment were generally associated with higher predicted risk. Poorer occupational functioning and productive work, specialist recommendation: treatment adjustment, higher disability severity level, and lower social participation were also associated with increased predicted risk.

‎

Exploratory Temporal Robustness Check

In the exploratory temporal robustness check, the LightGBM model was trained on the 2022 index-record cohort and evaluated on the 2023 index-record cohort using the same modeling pipeline as in the primary random 80/20 split analysis. The 2022 index-record cohort included 9028 patients, with a hospitalization-related severe relapse prevalence of approximately 19.7%. The 2023 index-record cohort included 30,790 patients, with a relapse prevalence of 6.3%. Among the 30,790 patients in the 2023 cohort, 21,566 had retained index records in the fourth quarter of 2023.

The model achieved a ROC-AUC of 0.93 and a PR-AUC of 0.62 in the 2023 index-record cohort, compared with 0.95 and 0.79, respectively, in the primary random 80/20 split analysis. At the default threshold of 0.50, relapse-class recall, precision, and F₁-score were 0.56, 0.63, and 0.60, respectively. When the decision threshold selected from the 2022 training cohort was applied, relapse-class recall increased to 0.61, with a precision of 0.60 and an F₁-score of 0.61.

These results suggest that the model retained discrimination when applied to records indexed in the subsequent calendar year. However, because the calendar year of the retained index record was partly influenced by the index-record selection rule, this analysis should be interpreted as an exploratory temporal robustness check (Table 4) rather than as evidence of conventional independent temporal validation. The lower PR-AUC and threshold-dependent performance in the 2023 cohort may partly reflect differences in relapse prevalence, case mix, and the index-record selection rule. Further details on quarter-wise performance and relapse prevalence are provided in Multimedia Appendix 2.

Discussion

Main Findings

Using large-scale, real-world community rehabilitation data from 39,818 individuals with schizophrenia recorded in the IMS, this study developed a hospitalization-related severe relapse risk identification model for community mental health management settings. The primary purpose of the model is not to provide precise predictions of relapse within a predefined future time window. Rather, it aims to use information routinely collected during community follow-up visits to dynamically identify patients who may have entered a high-risk trajectory and are more likely to require hospitalization-related intervention. Therefore, the model is intended to support risk screening, risk stratification, and early intervention within community mental health services.

The model integrated multidimensional information, including behavioral characteristics, functional status, interpersonal relationships, and community support, and used LightGBM together with SHAP-based interpretability analysis to identify key risk factors. The results indicated that medication persistence, specialist recommendation: medication adherence training, number of follow-ups, and age were among the most influential predictors of hospitalization-related severe relapse risk. In addition, educational attainment, occupational functioning and productive work, and social functioning assessment also demonstrated substantial importance, suggesting that functional recovery and social adaptation are closely associated with the risk patterns identified by the model [24].

Global feature importance analysis further highlighted the central role of medication management–related factors. Among them, medication persistence and specialist recommendation: medication adherence training were the 2 most influential predictors, while medication adherence also ranked among the key features. SHAP distribution plots showed that sustained medication use and good medication adherence were generally associated with lower risk, whereas intermittent medication use or refusal to take medication was more frequently associated with higher risk. These findings suggest that medication management–related factors play a critical role in the identification of hospitalization-related severe relapse risk. Previous studies have widely recognized medication nonadherence as one of the most important predictors of schizophrenia relapse [15,25], and medication education and supervision are typically associated with improved adherence and lower relapse rates [14,24]. Overall, the findings of this study are generally consistent with existing evidence. However, it is noteworthy that the proportion of patients recorded as having “regular medication adherence” was higher in the relapse group than in the nonrelapse group. For patients with more severe illness or higher relapse risk, community physicians and family members may implement stricter supervision and management, making these patients more likely to be documented as having “regular medication adherence” in follow-up records. In addition, this study found that better occupational functioning and productive work, better social functioning assessment, and better social participation were associated with lower model-predicted risk. This finding is consistent with previous research on social skills training, which has shown that social skills training and functional rehabilitation are typically associated with better social functioning, self-efficacy, and clinical stability [26,27]. Furthermore, Chen et al [28] found that higher levels of social participation are generally associated with lower levels of psychosocial stress and fewer social withdrawal behaviors [29,30].

Other Findings

Some findings in this study were inconsistent with previous literature. Higher educational attainment was associated with higher model-predicted relapse risk. One possible explanation is that patients with higher education levels may exhibit greater sensitivity to diagnostic labeling and social stigma, which may in turn be reflected in different illness perceptions and treatment attitudes during long-term management. This finding suggests that, in community rehabilitation settings, in addition to symptom control, greater attention should be paid to psychological adaptation, illness perception, and social support among higher-functioning patients. Future longitudinal studies are warranted to further explore the relationship between educational level, social functioning, and long-term disease management [31]. Higher disability severity was associated with higher model-predicted risk. This finding may be explained by patients’ current functional status, including impairments in self-care and social participation. Such functional impairments may increase perceived stigma and reduce self-esteem, thereby increasing patients’ vulnerability to relapse [32].

The present study also observed that younger age was associated with higher model-predicted relapse risk. Consistent with previous studies, younger patients or those with early-onset schizophrenia tend to have higher relapse rates and poorer long-term outcomes. On one hand, patients in the early stages of the illness often show more pronounced symptom instability and lower treatment stability. On the other hand, younger patients more frequently experience treatment discontinuation, loss to follow-up, and less stabilized social functioning. These characteristics have also been associated with higher relapse risk in previous research. In addition, younger patients are often in the process of establishing social roles, and greater pressures related to education, employment, and social adaptation may further complicate disease management [33].

Finally, a higher number of follow-ups was associated with higher model-predicted relapse risk. This finding may reflect the “high-risk intensive follow-up” pattern in community mental health management, whereby patients with unstable conditions, a frequent relapse history, or higher clinical risk receive more intensive community follow-up and health management. Therefore, higher follow-up frequency may reflect not only health care usage intensity but also indirectly indicate underlying disease severity and relapse risk.

Overall, dynamic management information within community mental health service systems may provide important complementary signals for relapse risk identification, further highlighting the potential use of machine learning models based on community EHRs in chronic mental disorder management.

Model Performance and Clinical Utility

In this study, LightGBM showed leading and balanced performance among the 6 evaluated models, with an ROC-AUC of 0.95, a PR-AUC of 0.79, a relapse-class recall of 0.77, a precision of 0.71, and an F₁-score of 0.74 on the independent test set. These results suggest that the model had good discriminative ability and performed reasonably well in identifying patients at risk of hospitalization-related severe relapse under a highly imbalanced outcome distribution. However, model performance should not be interpreted solely on the basis of ROC-AUC or overall accuracy. In community mental health management, the balance between sensitivity and precision is particularly important. A relapse-class recall of 0.77 indicates that the model identified approximately 77% of hospitalization-related severe relapse cases, while approximately 23% of such cases were not detected. Meanwhile, a precision of 0.71 suggests that most patients classified as high risk subsequently experienced hospitalization-related severe relapse, although false-positive alerts remained present. This trade-off has practical implications for community-based risk management. Lowering the decision threshold may improve the detection of high-risk patients, but it would also increase false-positive alerts, leading to additional follow-up visits, referral assessments, and intervention-related resource use. Conversely, using a stricter threshold may improve precision and reduce unnecessary alerts, but at the cost of missing more patients who may later require hospitalization-related intervention. Therefore, the model is better understood as a risk stratification and decision-support tool rather than a standalone screening instrument or a replacement for clinical judgment.

In community practice, the predicted risk score may be used to support follow-up prioritization and resource allocation. For example, patients with higher predicted risk may be considered for more frequent community follow-up, timely psychiatric reassessment, multidisciplinary review, or individualized management strategies addressing medication management, social functioning, family supervision, and rehabilitation participation. However, the model should not be used to automatically determine treatment adjustment, referral, or hospitalization. Final decisions should remain clinician-led and should integrate the predicted risk score with clinical assessment, patient history, family support, current mental state, and local service resources.

In addition, the slightly higher Brier score observed in the cost-sensitive LightGBM model indicates that improved relapse detection may be accompanied by a modest reduction in probability calibration. Thus, the predicted scores should be interpreted primarily as relative risk indicators for prioritizing community management, rather than as fully calibrated individual probabilities. Future studies should further optimize decision thresholds according to specific service scenarios, available community resources, and an acceptable false-positive burden, and should evaluate calibration and clinical utility before broader implementation.

The exploratory temporal robustness check further suggests that the model retained good discriminative performance when applied to a future cohort within the same community mental health management system. However, temporal robustness should be distinguished from external generalizability. Because all data in this study were derived from the Shanghai community mental health management system, model performance may still be influenced by local follow-up protocols, hospitalization referral criteria, documentation practices, patient characteristics, and community rehabilitation resources. Before broader implementation, the model should be externally validated using independent datasets from different geographic regions, health care systems, and community mental health service settings.

Finally, if the model is integrated into routine community mental health practice, its performance and calibration should be periodically monitored. As community management practices, follow-up routines, patient composition, service resources, and relapse prevalence evolve over time, recalibration or model updating may be required to maintain the model’s long-term applicability and clinical decision-support value.

Limitations

Several limitations should be acknowledged. First, this study did not use a fully standardized prediction window. For relapse cases, predictors were extracted from the last follow-up assessment before hospitalization; therefore, the model should be interpreted as a tool for identifying hospitalization-related severe relapse risk in real-world community management, rather than as a prognostic model for predicting relapse within a fixed future time horizon. Future studies should incorporate predefined prediction windows, longitudinal sequential data, and time-to-event modeling to evaluate earlier relapse risk detection. Second, the outcome was defined as hospitalization-related severe relapse, namely, psychiatric hospitalization due to acute exacerbation or deterioration of psychotic symptoms. This definition provides an objective and clinically meaningful end point, but it may not capture milder symptom exacerbations managed through outpatient care, medication adjustment, or intensified community follow-up. Third, generalizability remains limited because all data were derived from the Shanghai community mental health management system. Although the exploratory temporal robustness check using the 2022 and 2023 index-record cohorts suggested temporal robustness, the calendar year of the retained index record was partly influenced by outcome status because the index record was selected differently for nonrelapse and relapse patients. Model performance may also be affected by local follow-up protocols, hospitalization referral criteria, documentation practices, and related contextual factors. Future studies should prospectively validate the model in independently assembled cohorts or external datasets and consider periodic recalibration before broader implementation. In addition, some predictors, including specialist recommendations, medication-management variables, and follow-up-related variables, may partly reflect community management intensity or clinicians’ responses to perceived instability rather than patient-level risk factors alone. These variables should therefore be interpreted cautiously. The main analysis used integer encoding for categorical variables, which may impose artificial ordering on nominal variables, particularly for distance-based and linear comparator models. Future studies should consider one-hot encoding or model-native categorical feature handling to minimize the potential influence of encoding choices. Finally, several optimal hyperparameters were located at the boundary of the predefined search grid, suggesting that the search space may not have fully covered all potentially optimal parameter combinations. Therefore, the performance of each model should be interpreted as the performance achieved under the predefined tuning protocol rather than as the theoretical upper-bound performance of the algorithm. Future studies may use broader search ranges, random search, or Bayesian optimization for more exhaustive hyperparameter optimization.

Conclusions

Based on IMS data for community mental health services, this study developed a hospitalization-related severe relapse risk identification model for individuals with schizophrenia and used LightGBM in combination with SHAP analysis to enhance model interpretability. The study integrated multidimensional variables related to patient behavioral characteristics, social functioning, community management, and rehabilitation, and analyzed community follow-up records from 39,818 patients. The results showed that medication persistence, educational attainment, specialist recommendation for medication adherence training, age, and follow-up management were among the most influential predictors in the model. Specifically, sustained medication use and better occupational functioning and productive work were generally associated with lower predicted relapse risk, whereas higher educational attainment, high disability level, and younger age were more frequently associated with higher predicted relapse risk. Among the machine learning models evaluated, LightGBM demonstrated the leading and balanced performance in identifying relapse cases, suggesting that EHR-based machine learning approaches may provide a promising tool for the identification of hospitalization-related severe relapse risk in community mental health management settings.