Interpretable Machine Learning Model for Predicting Postpartum Depression: Retrospective Study

Introduction

Postpartum depression (PPD) is a common mental disorder characterized by low mood, loss of pleasure, and sleep disturbance during the postpartum period [1]. The prevalence of PPD ranges from 3% to 38% in different nations and is higher in limited-income countries [2,3]. PPD leads to adverse consequences for the mother and family members, such as emotional strain and increased caregiving burden. Women with PPD may experience prolonged periods of distress and are more vulnerable to recurrent depressive episodes [4]. Previous studies revealed that PPD can impair a mother’s parenting ability, such as breastfeeding, potentially resulting in enduring adverse effects on the child’s development across emotional, cognitive, and physical domains [5,6]. Moreover, PPD can strain family relationships and impose economic burdens due to increased health care needs and reduced productivity [6].

With such a profound impact, mothers should be routinely screened for PPD, and early interventions should be implemented. However, current screening for PPD is mainly based on existing depressive symptoms such as fatigue and sleep disturbance, which are believed to be overlooked due to overlap with normal physiological manifestations after delivery [7,8]. In addition, the diagnosis of PPD depends on patients’ subjective reporting of personal health conditions [9]. It is urgent to identify individuals with high risk for PPD before clinical symptoms appear, while no effective and validated screening tools are currently available [7,10].

Previous studies have identified several risk factors of PPD such as unplanned pregnancy, lack of social support, and family history of mental disorders [11,12]. However, limited variables in such studies led to a lack of integrity. Machine learning algorithms provide support for the development of predictive models to prevent and intervene adverse health outcomes, offering avenues for personalized prediction and intervention strategies [13,14]. Several studies have adapted machine learning into the prediction of PPD risk in the last few years and achieved impressive performance [15-17]. However, insufficient model explanations leave obstacles for actual implementation. Besides, mental disorders are strongly associated with cultural backgrounds and study populations. Thus, the challenge remains to develop more nuanced and culturally adaptable machine learning models for the early detection and effective management of PPD, bridging the gap in current research and practice.

Given the importance of early screening for PPD and the limitations mentioned earlier, we conducted a retrospective study at our institution. This study comprehensively collected variables from multiple aspects, adopted machine learning algorithms to identify risk factors, and aimed to achieve precise prediction of PPD.

Methods

Participants

Pregnant women who underwent perinatal examinations and delivered at West China Second University Hospital, Sichuan University, from January 2017 to December 2020 were invited to participate in this study. The study cohort was divided into training and validation sets by random sampling. Participants were screened for eligibility. The inclusion criteria were as follows: (1) participants who completed regular examinations and delivered at our institution, (2) participants with a gestational age of ≥28 weeks, and (3) participants who gave consent to participation and be followed up. The exclusion criteria were (1) participants with a psychiatric history in the 6 months before conception and (2) participants with missing data.

Outcome

Participants were assessed for PPD 3 months post partum with the Edinburgh Postnatal Depression Scale [18]. The Edinburgh Postnatal Depression Scale has 10 items concerning depressive symptoms, and each item is evaluated using scores ranging from 0 to 3, constituting a total score of 30. Participants who scored 13 or more were regarded as having PPD [18]. The diagnosis of PPD was confirmed by 2 experienced senior psychiatrists using the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) [19] and the Chinese Classification and Diagnostic Criteria of Mental Disorders, Third Edition (CCMD-3) [20].

Variable Screening

Demographic variables were collected from the electronic medical record system of our institution. Clinical variables were assessed and documented by qualified clinicians. Relevant laboratory indicators were collected at 28 weeks of gestation from the medical laboratory system of the institution.

Participants were assessed for antepartum depression before delivery with the Zung Self-Rating Depression Scale [21]. The Self-Rating Depression Scale is a self-reported scale with 20 items concerning depressive symptoms, and each item is evaluated with scores ranging from 0 to 4, according to the severity of symptoms. All participants with more than 53 points were regarded as having antepartum depression [22].

Social variables, including education, income, exposure to suspected adverse factors, and family and social relations, were collected using scales and self-administered questionnaires. Income level was assessed using the local minimum income standard. Suspected adverse factors included alcohol consumption and smoking. Family and social relations comprised spouses in good health, only child, planned pregnancy, social support, family satisfaction, adverse marital status, and family history of mental illness. The level of social support was measured using the Social Support Rating Scale, which is widely used to assess social support with great reliability [23]. Scores higher than 35 are considered normal; scores of 35 or lower indicate low levels of social support [24]. Family satisfaction was assessed using the Family Adaptation, Partnership, Growth, Affection, Resolve index [25]. The Family Adaptation, Partnership, Growth, Affection, Resolve index consists of 5 items, each with a score ranging from 0 to 2. It systematically evaluates the level of family care a pregnant woman receives. A total score of 0‐3 represents a low level of family satisfaction, and a score higher than 4 represents a normal level.

To avoid the potential bias of multicollinearity and overfitting, least absolute shrinkage and selection operator (LASSO) regression was performed to select and filter the variables in the training set. LASSO is a regression-based methodology that can reduce model complexity; multicollinearity and overfitting are avoided by constructing a penalty function [26]. LASSO regression is applied to filter a large number of variables and remove those that are insignificant [27,28]. The 5-fold cross-validation method was used to calculate the optimal λ values, and variables with nonzero coefficients were selected as the final predictive factors. After LASSO regression, the variance inflation factor (VIF) was calculated among the included variables to assess multicollinearity. The VIF was introduced to understand the impact of collinearity in regression models and has since been widely applied in various fields, including medical research [29-31]. VIF helps ensure that machine learning models or statistical models are not adversely affected by collinear predictors [32]. Typically, a VIF value greater than 10 is considered indicative of high multicollinearity, which may necessitate removing or adjusting variables to improve model stability [29,33].

Model Development

We used the following 3 machine learning algorithms to develop the PPD prediction model: extreme gradient boosting (XGBoost), random forest (RF), and gradient boosting machine (GBM). XGBoost is a powerful and efficient machine learning algorithm known for its exceptional performance in regression, classification, and ranking problems. It is an extension of the traditional gradient boosting method that combines multiple weak classifiers to create a strong classifier that minimizes the loss function [34]. RF is an ensemble machine learning algorithm based on decision trees. It creates multiple decision trees, each based on a randomly sampled subset of the training data to create a more accurate and robust output [35]. GBM is a popular machine learning algorithm that combines the principles of boosting and gradient descent to create a powerful predictive model [36]. Additionally, logistic regression, a traditional method, was implemented to predict PPD as a control.

Machine learning models were developed in the training set. To mitigate overfitting and achieve ideal model performance, hyperparameters for each machine learning model were tuned by grid search. In each session of hyperparameter tuning, 3-fold cross-validation was implemented, and the area under the receiver operating characteristic curve (AUC) was the criterion to assess model performance [37]. The combination of hyperparameters with the largest AUC value was further evaluated in the validation set.

Model Evaluation

Predictive models were evaluated with the receiver operating characteristic curve (ROC) and decision curve analysis (DCA) in the validation set. ROC reflects the ability of a model to discriminate PPD [38]. DCA is used to evaluate and compare the clinical utility of different diagnostic or predictive models. It provides a framework for assessing the net benefit of a model by taking into account the potential harms and benefits associated with different decision thresholds [39]. Additionally, accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of each model were calculated for comprehensive evaluation. Based on ROC, the predictive model with the greatest AUC value was considered as the optimal model, which would be further explored for interpretation.

Model Interpretation

We performed variable importance, partial dependence plot (PDP), and Shapley Additive Explanation (SHAP) to interpret the optimal predictive model. Variable importance assesses the contribution of each input variable by calculating the decrease in error when split by a variable [40]. PDPs calculate the partial dependence of a variable by fixing the values of other variables and observing the variation in the outcome [41]. It helps to explain how the outcome changes with changes in input variables. SHAP measures the contribution of variables in each individual sample [42]. The SHAP values show how much each variable contributes, either positively or negatively, to the outcome.

Statistical Software

All statistical analyses were performed with R software (version 4.3.1; R Foundation for Statistical Computing). LASSO regression was performed using the R package glmnet. Logistic regression model was implemented using the R package glm. XGBoost, RF, and GBM models were developed and assessed with R package mlr3. Model interpretation was performed with R packages fastshap and pdp. Other involved packages include xgboost, randomForest, gbm, pROC, ggplot2, and their various dependencies.

Ethical Considerations

This study was approved by the ethics committee of West China Second University Hospital, Sichuan University (approval 2021-186). Informed consent was obtained from all individual participants involved in the study. The original informed consent covered the secondary use of the data without the need for additional consent. All participant data were anonymized to protect privacy and confidentiality. No compensation was provided for participation in this study.

Results

Participant Characteristics

The overall procedure of this study is shown in Figure 1. After eligibility screening, 2055 participants were included in the study cohort. A total of 78 variables were incorporated in our study including 16 psychosocial characteristics, 43 obstetric characteristics, and 19 laboratory indicators. The baseline characteristics were analyzed by χ² test and Wilcoxon test for category variables and continuous variables, respectively. The detailed characteristics are shown in Tables 1-3. Of these participants, 697 (33.9%) participants were diagnosed with PPD, 621 (30.2%) participants had antepartum depression, 101 (4.9%) were unemployed, 187 (9.1%) had an income below the local minimum income standard, 1947 (94.7%) pregnant women were the only child of the family, and 45 (2.2%) reported low family satisfaction; the median age of participants was 32 (IQR 29-35) years.

‎

Variable Screening

After LASSO regression, 18 variables with nonzero coefficients were identified as potential predictors of PPD. Among these variables, the 5-minute Apgar (appearance, pulse, grimace, activity, and respiration) score and the 10-minute Apgar score had VIF values over 10, indicating multicollinearity between them. Of these 2 variables, the 5-minute Apgar score had a lower coefficient in absolute value in the LASSO regression, suggesting its lower contribution to the outcome; therefore, it was excluded. Another round of VIF analysis was performed after excluding the 5-minute Apgar score, and the results showed that all remaining variables had a VIF below 10, indicating low multicollinearity. Finally, 17 variables including prenatal depression, ethnics, occupation, income, only child, family satisfaction, adverse marital status, amniotic fluid volume disorder, Rh negative, intrauterine death, fetal distress, age, fetal weight, 10-minute Apgar score, serum ferroprotein, thyroid-stimulating hormone (TSH), and thyroid peroxidase antibody (TPOAb) were identified as features to develop predictive models. The detailed results of the LASSO regression and VIF analyses are presented in Table 4.

Model Development and Evaluation

Predictive models were established in the training set. With optimal hyperparameters, models were evaluated in the validation set. The AUC values obtained in the validation set of all 4 models were above 0.75 (Table 5 and Figure 2A). The XGBoost model outperformed other models with the highest AUC of 0.849 (95% CI 0.828‐0.871). GBM had the poorest performance with an AUC value of 0.779 (95% CI 0.738‐0.820). Detailed results of model evaluation were shown in Table 5.

‎

DCA (Figure 2B) was performed for 4 models in the validation set to compare the net benefit of the best model and alternative approaches for clinical decision-making. Treatment strategies informed by any of the 4 models are superior to the default strategies of treating all or no patients. The net benefit of the XGBoost model exceeded those of the other models at 20%-60% threshold probabilities.

Model Interpretation

The XGBoost model, identified as the optimal model in terms of AUC value, was further explored for interpretation. Table 6 demonstrates the variable importance of the XGBoost model. Antepartum depression, TSH, fetal weight, serum ferritin, TPOAb, and age were the 6 variables that most influenced the outcome of the model.

PDPs illustrate a visual representation of the relationship between the most influential variables and the predicted response while accounting for the average effect of the other predictors in the model (Figure 3). As our predictive outcome is a binary categorical variable, the impacts of variables on the outcome were presented in the form of predictive probability ranging from 0 to 1. These plots indicate that the probability of PPD increases when participants have antepartum depression, higher TSH, higher serum ferritin, and older age. Likewise, the probability descends when participants have higher fetal weight and higher TPOAb.

‎

SHAP provides an insight into how variables influence the prediction in each single sample (Figure 4). It can be concluded that the risk of PPD increases for participants with antepartum depression, lower fetal weight, lower level of TPOAb, elevated serum ferritin, and older age. The overall impact of TSH is not obvious. The interpretation of SHAP was mostly consistent with the interpretation of PDP. A higher risk of PPD is also associated with lower Apgar scores at 10 minutes, low income, amniotic fluid volume disorder, fetal distress, unsatisfactory family conditions, only child in the family, intrauterine death, Rh negative blood type, adverse marital status, and other ethnics.

‎

Discussion

Principal Findings

This study developed and validated a machine learning–based model for prediction of PPD with an AUC of 0.849. Through the model interpretation of our optimal XGBoost model, several significant predictors of PPD were identified. These findings derived from the XGBoost model provide insightful contributions to the understanding of PPD.

Based on variable importance, antepartum depression was the most influential predictor of PPD in our analysis. Women with antepartum depression are likely to extend depressive symptoms into the postpartum period. In an epidemiology study, more than 54% of women with PPD reported depressive symptoms during pregnancy [5]. Several studies excluded participants with antepartum depression to focus on newly diagnosed PPD [16,43]. This might avoid the bias associated with chronic depression but neglect the impact of maternal prenatal mental status on PPD.

In addition, our study identified key biochemical markers of PPD including TSH, TPOAb, and serum ferritin. According to the result of the PDP interpretation, women with elevated serum ferritin levels were prone to PPD. Ferritin serves a critical role in the synthesis of monoamine neurotransmitters including dopamine [44]. With excessive ferritin, neurotransmitter dysregulation might play a part in the onset of PPD [45]. This finding warns the risks of excessive iron supplementation during pregnancy. Apart from that, elevated TSH and declined TPOAb were associated with a higher risk of PPD in our model interpretation. Elevated TSH often indicates hypothyroidism, whose clinical manifestation includes depression [46]. TPOAb is an autoantibody against the enzyme thyroid peroxidase and is commonly associated with autoimmune thyroid diseases such as Hashimoto thyroiditis [47,48]. A systematic review reported that the association between TPOAb and PPD remains controversial [49]. The specific mechanism of how TPOAb affects PPD requires more investigation.

Additionally, women with older age and lower infant weight were prone to PPD in our findings. This aligns with existing literature [49]. Other commonly recognized predictors like lack of social support, gestational diabetes, and overweight were excluded in the LASSO regression due to potential multicollinearity. These results not only validate some of the existing hypotheses about the pathophysiology of PPD but also open new avenues for research, particularly in the context of developing more effective, holistic screening methods.

Our study offers several advancements over previous research on PPD prediction. Earlier studies predominantly focused on clinical factors, such as obstetric history and comorbidities during pregnancy [50-52]. Our research comprehensively collected variables from multiple domains, including clinical, psychosocial, and biochemical markers. This broader scope allows for a more holistic view of the potential risk factors contributing to PPD, addressing the multifactorial nature of the condition. In addition, many previous studies were limited by relatively small sample sizes, often involving fewer than 1000 participants, which may have restricted the generalizability of their findings [50,51,53]. In contrast, our study included a larger cohort of over 2000 participants, providing greater statistical power and a more reliable basis for identifying significant risk factors. This larger sample size also enhances the model’s ability to detect more subtle associations between variables and PPD, which might have been overlooked in studies with smaller sample size. Moreover, previous research has often been limited to populations in Western countries [54,55]. Our study focuses on a Chinese cohort, offering insights that are culturally specific and potentially more relevant for addressing PPD in non-Western settings. This regional focus helps fill a critical gap in the literature, as the risk factors and prevalence of PPD may vary significantly between different cultural and geographical populations. Finally, our inclusion of biochemical markers, such as thyroid function and serum ferritin levels, adds a novel dimension to PPD research. These physiological indicators have rarely been incorporated in prior studies; yet, they may play a crucial role in understanding the biological underpinnings of PPD. By integrating these markers with traditional clinical and psychosocial factors, our study provides a more comprehensive framework for early detection and intervention.

Although our study was conducted on a specific population from Southwestern China, the comprehensive nature of the variables included in the model suggests that it could be adapted to other populations with similar characteristics. Future studies could explore the model’s applicability in different regions and cultural contexts, potentially leading to a robust, universally applicable tool for PPD prediction.

Our study offers significant insights into the clinical application of machine learning models for PPD. By integrating a broad spectrum of both biochemical and psychosocial factors, our models offer a more nuanced and accurate prediction compared to traditional methods. Incorporating often-overlooked indicators such as thyroid function and iron metabolism provides insight into the early screening of PPD. The XGBoost model, which demonstrated the highest performance, is particularly valuable for its ability to manage complex interactions between variables, making it universally applicable in a clinical setting where multiple risk factors are at play. This comprehensive approach enhances the understanding risk factors for PPD and supports more effective early interventions. Future research can build on these findings by validating them in larger, more diverse populations, integrating these predictive factors into routine prenatal care, and exploring interventions targeting these risks to reduce the incidence of PPD.

Limitations

This work is not without limitations. First, our analysis is constrained by the retrospective nature of the data, which may introduce biases such as recall bias or selection bias. Additionally, during the process of variable screening, removing variables with collinearity ensured the independence of final predictors, but also eliminated substantial amounts of variables, leading to loss of information. Furthermore, the study’s reliance on a specific population from a single institution may limit the generalizability of our findings to broader, more diverse populations.

Conclusions

This study developed and validated several machine learning–based models for predicting PPD, integrating a comprehensive set of clinical, psychosocial, and biochemical factors, and incorporating a larger sample size. The XGBoost model was considered as the optimal model with an AUC of 0.849. Interpretation derived from the predictive model revealed significant predictors of PPD, encompassing antepartum depression, elevated TSH, declined TPOAb, elevated serum ferritin, older age, and lower infant weight. These identified risk factors could be implemented to the early screening of PPD for individuals at high risk. These findings underscore the advantages of integrating diverse predictors and advanced machine learning techniques to improve early screening for PPD. This approach not only enhances prediction accuracy but also provides valuable insights for future research and clinical applications.