Machine Learning and Shapley Additive Explanations Value Integration for Predicting the Prognostic of Anti-N-Methyl-D-Aspartate Receptor Encephalitis: Model Development and Evaluation Study

Introduction

Anti-N-methyl-D-aspartate receptor (NMDAR) encephalitis represents more than 80% of autoimmune encephalitis (AE) cases. Its pathogenesis is closely linked to synaptic plasticity impairment mediated by autoantibodies targeting the NMDAR GluN1 subunit [1,2]. These antibodies induce limbic system dysfunction and characteristic neuropsychiatric manifestations [3]. While first-line immunotherapy has markedly improved survival rates, up to 30%‐47% of patients continue to experience cognitive deficits, psychiatric symptoms, or functional disabilities [4-6]. Consequently, the development of a dynamic prognostic evaluation system is critical for guiding individualized treatment strategies.

Since the initial report of NMDAR encephalitis in 2007, researchers have investigated the prognostic value of various biomarkers. Current evidence indicates that cerebrospinal fluid antibody titers, extreme delta brush on electroencephalogram, and acute-phase inflammatory markers may correlate with disease severity [5,7-9]. However, single-dimensional biomarkers exhibit insufficient predictive power. Currently, there is a lack of integrated multidimensional data for constructing models to predict patient outcomes. Existing prediction models [10-12] predominantly rely on the modified Rankin Scale [13] and are limited in assessing the cognitive-mental dimension of NMDAR encephalitis. Moreover, traditional logistic regression approaches fail to adequately analyze complex nonlinear interactions among predictors, thereby reducing their clinical applicability.

Machine learning (ML) offers transformative potential for integrating and analyzing complex clinical data. In contrast to traditional statistical methods, ML algorithms can identify intricate feature interactions and enhance predictive performance in small-sample rare disease cohorts [14]. This capability has been successfully demonstrated in neurodegenerative diseases and tumor prognosis prediction by enhancing the feature representation of small-sample datasets. However, no study has systematically applied ML for prognostic prediction in NMDAR encephalitis.

The objective of this study is to integrate clinical features, neuroimaging findings, serological markers, and Clinical Evaluation Scale for Autoimmune Encephalitis (CASE) [15] scores to construct a prognostic prediction model for NMDAR encephalitis based on ML and identify early prognostic signatures through Shapley Additive Explanations (SHAP)–driven feature interpretability. Furthermore, a web-based clinical tool was developed to deploy the model, enabling clinicians to use it directly for prognostic predictions and to plan interventions (eg, escalation to second-line therapy or multiple immune combinations within a critical 4-week postsymptom window).

Methods

Study Design

Data for this retrospective study were collected from the Third Affiliated Hospital of Sun Yat-sen University from 2015 to 2024. This study was conducted in accordance with the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines [16]. Informed consent was obtained from all participants. This study was approved by the ethics committee of the Third Affiliated Hospital of Sun Yat-sen University (II2024-370-01). This study comprised 3 phases: (1) cohort selection, (2) variable optimization and model development, and (3) validation and clinical translation tools.

Study Population

All patients underwent comprehensive evaluations, including neuroimaging, electroencephalogram, cerebrospinal fluid (CSF) analysis, and tumor screening.

Eligibility Criteria

The inclusion and exclusion criteria are listed in Textbox 1.

Data Collection

Data were extracted from electronic medical records. Standardized variables included the following:

1. Demographics: age at disease onset, gender, occupation, and BMI.
2. Clinical characteristics: (1) prodromal symptoms (fever, headache, and dizziness) and (2) initial symptoms (psychiatric abnormalities, seizures, altered consciousness, and memory deficits).
3. Critical care interventions: intensive care unit (ICU) admission and mechanical ventilation requirement.
4. Laboratory parameters: CSF or serum anti-NMDAR antibody titers. CSF samples were analyzed using a cell-based assay by a trained specialist in a laboratory with experience in interpreting these assays. (1) Sex hormone tests include luteinizing hormone, follicle-stimulating hormone, estradiol, progestogen, and testosterone. (2) Hematological and biochemical analyses: white blood cell count, neutrophils, lymphocytes, monocytes, uric acid (UA), erythrocyte sedimentation rate, C-reactive protein, and high-density lipoprotein cholesterol.

1. Neuroimaging: Baseline magnetic resonance imaging (MRI) or electroencephalogram findings, including lesion quantity, and MRI. Analysis was performed by a senior neuroradiologist and neurologist using 1.5/3.0 Tesla scanners with T1-weighted, T2-weighted, and FLAIR sequences. All electroencephalogram data were recorded according to the international 10‐20 system using an electroencephalogram Digi Track (ELMIKO) with 19 electrodes. Tumor details (if applicable) included tumor size (mean of longitudinal and transverse diameters on imaging) and number of tumors.

Evaluation Prognosis and Definitions

Outcome metric and clinical severity were quantified using the CASE scale, a validated 9-domain instrument assessing (seizures, memory deficits, psychiatric symptoms, consciousness, speech, motor or dystonia, gait or ataxia, brainstem dysfunction, muscle strength; total score: 0‐27). Prognostic stratification was defined as favorable: 0‐4 (minimal residual symptoms), moderate: 5‐9 (partial functional impairment), and poor: 10‐27 (severe disability). Lower CASE scores at follow-up indicated improved functional recovery.

Data Preprocessing and Feature Engineering

Categorical variables were coded by labels’ encoding, while continuous variables were standardized using the Standard Scaler to mitigate feature scale bias. From an initial pool of 47 candidate predictors (selected via systematic literature review and clinical consensus), recursive feature elimination (RFE) with random forest (RF) (500 trees, Gini impurity) was applied. Features exceeding the importance threshold (P>.01) were retained. The threshold of 0.01 was empirically validated as optimal for maximizing cross-validation accuracy. This value consistently yielded peak performance in predicting prognosis across all candidate models (decision tree [DT], RF, extreme gradient boosting [XGBoost]), balancing feature retention with model generalizability. To rigorously mitigate randomness in RFE with a cross-validation workflow, we implemented a dual-strategy approach: (1) stratified cross-validation. Instead of relying on a single random train-test split, we used stratified 5-fold CV throughout the RFE process. (2) To test the stability of the selected feature set, we repeated the RFE with a cross-validation process multiple times with different random seeds (random_state=42). We then analyzed the frequency with which each feature was selected. Features that consistently appeared across multiple runs were considered more robust and were prioritized.

Model Development and Validation

The dataset was strategically partitioned through random stratified sampling into a training set (80%, n=112) and a validation set (20%, n=28). The training set was exclusively used for model development, while the validation set served as an independent cohort for performance evaluation. The 3 ML algorithms, including RF, XGBoost, and DT, were used to construct the prediction model of NMDAR. Benchmarking is an important way to systematically evaluate and compare the performance of ML models. This process involves evaluating multiple models on standardized datasets and using consistent evaluation metrics to ensure fair comparisons. In this study, model performance was assessed using mean squared error (MSE), root-mean-squared error (RMSE), coefficient of determination (R²), mean absolute error, and mean absolute percentage error. The optimal model was selected based on minimal RMSE in validation. A 5-fold cross-validation was used to verify the stability of the model (Table 1).

Model Interpretation and Network Application

The ML has difficulty explaining the contribution of each feature due to its black-box principle. Interpretability denotes elucidating how ML models produce results. To elucidate feature contributions, SHAP values were used to quantify and interpret feature importance in the best-performing model. The algorithm provides a measure of feature importance across the model. Local interpretability was visualized via force plots, while global patterns were assessed through mean |SHAP values|.

Network Application

Network application was deployed using Flask 2.3.2 backend with REST API, dynamic input forms aligned with selected predictors, real-time SHAP plot generation, and risk stratification output (probability scores with 95% CIs). This tool operates independently from any model and uses input predictor variables for result interpretation while enabling predictions for individual cases.

Statistical Analysis

The Shapiro–Wilk normality test was performed to assess the data normality. Continuous variables are reported as medians with interquartile ranges (IQRs) for skewed distributed variables. Categorical variables are reported as frequency and percentage.

All statistical analyses were performed using IBM SPSS Statistics 22 (SPSS Inc). The predictive model construction and graphical representations were implemented using Python V3.7 (Python Software Foundation) and Prism 10.0 (GraphPad Software), respectively.

Ethical Considerations

The Ethics Committee of the Third Affiliated Hospital, Sun Yat-sen University (II2024-370–01) gave ethical approval on December 30, 2024. Informed consent was waived by the committee, as the study involved retrospective analysis of anonymized data and did not include any identifiable personal information. All procedures were conducted in accordance with the ethical standards of the institutional and national research committees and with the principles outlined in the Declaration of Helsinki. We ensured that the privacy and confidentiality of all participants were strictly maintained; data were fully anonymized before analysis, and no individual-level identifiers were collected, stored, or reported. No compensation was provided to participants, as the study did not involve direct interaction with human participants.

Results

A retrospective cohort of 257 patients diagnosed with anti-NMDAR encephalitis was identified from the electronic medical records of the Third Affiliated Hospital of Sun Yat-sen University (January 2014-November 2024). According to the inclusion and exclusion criteria, 140 patients were included in the final analysis (Figure 1).

‎

Population Characteristics

Among the final cohort (median follow-up 8, IQR 2‐23 months), 102 of 140 (72.9%) were female patients, with a median age of 22 (IQR 17‐30) years; of these, 50 (35.7%) were students. Pulmonary infections emerged as the most frequent comorbidity, occurring in 54 of 140 (38.6%) patients, whereas fever was the leading prodromal symptom, appearing in 41 (29.3%) patients. NMDAR encephalitis–associated diffuse slow waves and delta brush patterns in electroencephalography were observed in 10% (14/140) and 2.1% (3/140) of cases, respectively. Among the 140 patients, 30 (21.4%) had tumors (28 benign teratomas and 2 malignancies). Median acute-phase CASE score was 11 (IQR 6‐16), improving to 0 (IQR 0‐1) at 1-year follow-up (Figure 2). Full demographic and clinical characteristics are detailed in Table 2.

‎

Variable Screening

This study examined predictors that longitudinally affect disease severity in the anti-NMDAR encephalitis observational cohort. The CASE score provides more insight into specific AE-related symptoms at onset, allowing for longitudinal assessment of these symptoms [17]. The predictor variables identified by RF were selected as common elements in the development of the prediction model. Select the importance threshold (P>.01) in the training set; these features were included in our study to construct a reduced feature set (selected features). A total of 20 of the 47 features were selected for inclusion in the ML model: 4 demographic characteristics (age, gender, BMI, and occupation), 3 clinical characteristics (admission to ICU, mechanical ventilation, initial symptoms-memory deficits), 11 laboratory parameters (UA, erythrocyte sedimentation rate, C-reactive protein, high-density lipoprotein cholesterol, CSF antibodies, neutrophils, estradiol, white blood cell, progestogen, and testosterone), and 2 neuroimaging characteristics (number of MR lesions and tumor size). The association between each candidate predictor is shown in Multimedia Appendix 1.

Model Development and Performance Comparison

The prognostic performance of the selected features was systematically evaluated using 3 ML algorithms: RF, XGBoost, and DT. As detailed in Table 3, the RF model demonstrated superior discriminative capacity across all evaluation metrics (mean squared error=11.01, RMSE=3.32, R²=0.71, mean absolute error=2.49, and mean absolute percentage error=0.48). The RF model, selected as the optimal prognostic tool based on minimal RMSE, showed advantages in predicting functional outcomes of NMDAR encephalitis (the model parameters are shown in Table 4). A 5-fold cross-validation was used to verify the stability of the model (Table 1).

Interpretation of the Model

To elucidate feature contributions and interpret model predictions, we used SHAP analysis, a robust game-theoretic approach that quantifies the relative importance of each predictive feature (Figures 3 and 4). SHAP values represent the magnitude and directionality of each predictor’s influence on outcome probability, where positive values indicate increased disability risk. This visualization demonstrates how key features interact to determine individualized prognosis, enabling clinicians to prioritize modifiable risk factors. The summary plot (Figure 5) uses personalized feature attribution to represent range and distribution characteristics. We can directly see the impact of each feature on the prediction of prognostic features in NMDAR. Red to blue represents the eigenvalue from high to low. The thickness of the line represents the sample distribution.

‎

‎

‎

In the optimal performing RF model, the top 3 predictive features for prognostic predictors of NMDAR were admission to ICU (mean |SHAP value|=1.65), initial symptoms-memory loss (0.69), and UA (0.53). Other significant predictors included occupation, cerebrospinal fluid antibody titer, lymphocytes, sex, age, tumor size, BMI, estradiol, erythrocyte sedimentation rate, high-density lipoprotein cholesterol, need for assisted ventilation, number of MR lesions, white blood cell count, neutrophil count, C-reactive protein, progestogen, and testosterone.

Development of Convenient Applications

This platform features an intuitive graphical interface that accepts clinical input of 20 validated predictive variables, subsequently performing automated calculation of composite prognostic scores for patients with anti-NMDAR encephalitis through embedded algorithmic processing. Real-time risk stratification results provide clinicians with a quantitative prognostic assessment to support evidence-based treatment decisions. This decision support tool optimizes the process of translating predictive analytics into clinical practice (Figure 6).

‎

Discussion

Principal Findings

We developed 3 ML-based prognostic models (XGBoost, RF, and DT) for anti-NMDAR encephalitis. A total of 20 pivotal predictors spanning demographic, clinical manifestations, laboratory parameters, and neuroimaging characteristics were identified from 47 candidate variables. The RF model exhibited the best performance. The SHAP interpretability analysis revealed that ICU admission (mean |SHAP value|=1.65), memory deficits at onset (0.69), and UA (0.53) were the most important predictive factors. To our knowledge, this is the first predictive model for anti-NMDAR encephalitis prognosis developed using ML.

Anti-NMDAR encephalitis is an autoimmune disease associated with antibodies targeting the GluN1 subunit of the NMDAR. Although it is the most common AE in adults, it is still considered a rare disease. Due to the widespread geographic distribution of patients, phenotypic heterogeneity, and insufficient understanding of the pathophysiological mechanisms, there are still many challenges in the diagnosis and treatment of this disease [18]. The outcomes for patients with anti-NMDAR encephalitis vary widely, with approximately 75% of patients achieving complete recovery or having only mild sequelae, while the remaining 25% experience severe neurological deficits or death [5,19]. Identifying patients at risk of severe disease progression and persistent deficits at an early stage is of great clinical significance, as it can inform individualized therapeutic decisions and facilitate timely escalation of treatment. Currently, there is a lack of quantitative predictive models for assessing disease prognosis.

In recent years, various ML algorithms have been widely applied in the field of medicine for treatment and prognosis prediction. Compared with traditional statistical methods, such as logistic regression, ML can directly capture nonlinear and complex interactions. Algorithms such as RF and light gradient boosting machine can automatically identify the most important predictive features, reducing the subjectivity of manual feature selection.

Current prognostic tools face critical challenges: (1) they are focused on single-indicator predictions, and traditional statistical methods were used for analysis. For example, Kashyap et al [20] used the neutrophil-to-lymphocyte ratio (NLR) for prognostic prediction in pediatric anti-NMDAR encephalitis, while Lin et al [11] used serum neuron-specific enolase for prognostic prediction. Such single-indicator prediction methods have obvious limitations in a rare disease with diverse manifestations and complex disease courses. (2) They rely on modified Rankin Scale that inadequately captures cognitive or psychiatric dimensions [12,21]. The modified Rankin Scale is a disability scale developed for stroke clinical trials and largely focuses on motor dysfunction. Such disability scales are limited in capturing nonmotor disabilities in AE. (3) They rely on delayed assessments, such as the anti-NMDAR encephalitis one-year functional status score [22], which requires a 4-week observation period and therefore lacks timeliness in prognostic scoring. For patients with poor prognosis who need timely treatment escalation or combination therapies, this delay may result in missed early therapeutic windows. (4) They depend on complex biomarkers requiring specialized processing: Xiang et al [10] developed a fusion prediction model based on 4 MRI sequences (T1WI, T2WI, FLAIR, and DWI) for early prediction of functional outcomes in adult patients with anti-NMDAR encephalitis. However, these indicators are not easily obtainable and require specialized radiologists to extract data, which severely limits clinical translation and application and is not conducive to clinical implementation.

We used 47 clinical variables available in the electronic medical records of a large tertiary hospital. The selection of these 47 variables was based on the following criteria: first, the aim of this study was to develop a practical prediction application for clinical settings; therefore, we needed to choose patient indicators that are easily accessible to clinicians. Second, according to previous literature and meta-analysis results, some variables were confirmed to be significantly related to patient prognosis. We also made certain improvements. Previous analyses of the predictive value of clinical MRI were based on binary assessments of normal or abnormal MRI. In this study, we used the number of lesions in MRI for analysis, which is more accurate. In feature selection, RFE with RF was applied. Features exceeding the importance threshold (P>.01) were retained. Ultimately, 20 influential variables were selected to develop a novel ML model to assess the prognosis of anti-NMDAR encephalitis.

For the outcome metric, CASE [15] is a disease severity scale specifically designed for AE and can clearly display the clinical trajectory from the acute phase to the recovery phase, making it suitable for longitudinal assessment of patient outcomes during treatment. The CASE score has not yet been used in prognostic model construction. Using the CASE score as the prognostic outcome variable in our study makes the prognostic assessment of patients more scientific and comprehensive.

We chose RF, DT, and XGBoost to construct predictive models for anti-NMDA encephalitis prognosis and evaluated the distinguishing features of each model through benchmark testing (mean squared error, RMSE, R², mean absolute error, and mean absolute percentage error) to determine the most suitable model for prediction. Ultimately, we selected the best-performing RF model and further deployed it on a web platform.

In our study, SHAP visualization analysis confirmed that admission to the ICU (mean |SHAP value|=1.65), initial symptoms of memory deficits (0.69), and UA levels are the main predictive factors for prognosis. In addition, other clinical features such as the presence of memory deficits, tracheostomy, and CSF antibody titers were linked to poorer prognoses in anti-NMDAR encephalitis.

In a retrospective study of 382 patients with anti-NMDAR encephalitis, Balu et al [23] identified ICU admission as the most significant univariate predictor [23]. A previous systematic study also revealed associations between decreased consciousness, ICU admission, and poorer outcomes in anti-NMDAR encephalitis [7]. Conversely, the binary risk stratification by ICU admission underscores the need for intensive monitoring protocols at disease onset. This is consistent with our result. SHAP analysis indicated that serum UA concentration plays a crucial role in predicting anti-NMDAR encephalitis prognosis, aligning with the findings of Liu et al [11]. SHAP dependence analysis revealed that UA values >400 μmol/L sharply reduced CASE probability (SHAP<−1.0). As a central nervous system autoimmune inflammatory disease, the pathogenesis of anti-NMDAR encephalitis remains unclear but is known to involve inflammatory responses. UA reduces proinflammatory cytokine release and removes reactive oxygen species, protecting the body from oxidative stress [24]. UA has also been reported to exert neuroprotective effects in various neurological diseases, such as Alzheimer disease, Parkinson disease, and multiple sclerosis [25]. Thus, UA likely serves as a protective factor in the course of anti-NMDAR encephalitis.

Refractory patients may benefit from alternative drugs such as tocilizumab, bortezomib, or daratumumab [10,18]. Our novel model, developed through rigorous variable selection, development, validation, interpretation, and web deployment, demonstrates excellent predictive accuracy. A significant advantage is that all predictive factors used in the model are easily obtainable by clinicians, making risk stratification within 24 hours, enabling them to promptly identify patients with poor prognoses and guide more aggressive treatment escalation to prevent mild cases from progressing to treatment-resistant severe encephalitis.

Strengths and Limitations

The strengths of this study lie in three advancements: (1) We developed a novel ML model for prognostic prediction in encephalitis, (2) this model is exclusively based on clinically accessible parameters, and (3) we created a visualized web-based decision support platform to enable real-time clinical implementation. Our study also has several limitations. First, due to the rarity of the disease, the sample size was relatively small, and it is impossible to conduct subgroup analysis and statistics for each clinical manifestation. Second, the data used for model development were obtained from patients admitted to a specific medical institution rather than from a large database, which may introduce bias. Future research should consider training and validating the model across different national contexts.

Conclusions

In this study, we developed and validated an interpretable ML model based on the RF algorithm. The model demonstrated high predictive accuracy, with robust performance metrics, supporting its potential utility in clinical settings. Through SHAP analysis, key prognostic predictors were identified, including admission to ICU, memory deficits at initial presentation, and serum UA levels, offering novel insights into risk stratification. This tool may assist clinicians in making evidence-based decisions, optimizing interventions, and ultimately improving outcomes for patients with anti-NMDAR encephalitis.