Data were extracted from the ART EMRs covering the period from 2005 to 2024. Only patients on ART for ≥6 months before analysis were included to ensure baseline predictor availability.

This study was conducted at public health facilities in Gondar City Administration (Amhara Region, Ethiopia) from October 10 to December 10, 2024. Gondar City (population 457,938) is located 748 km northwest of Addis Ababa. In the Gondar City Administration, several health care facilities provide ART services to people living with HIV. Among them, the University of Gondar Specialized and Comprehensive Hospital serves as a key provider of ART services, offering specialized care to patients. Additionally, multiple health centers, including Azezo Health Center, Gondar Health Center, Maraki Health Center, Mintwab Health Center, St. Gebriel Health Center, Teda Health Center, and Woleka Health Center, also provide ART services, ensuring accessibility to treatment at various levels of health care.

Health facility mapping was conducted to provide context to the data source; however, facility service gaps were not the focus of this predictive modeling study (see Figure 1).

The retrospective cohort study analyzed baseline-only EMR data (ART initiation records) to predict prospective mortality (see Figure 2).

Data are sourced from EMR-ART databases across Gondar City Administration public health facilities, containing baseline sociodemographic, clinical, laboratory, and treatment initiation data.

Ethical approval for this study was obtained from the College of Medicine and Health Sciences Institutional Research Ethics Review Committee (CMHS IRERC) at Debre Markos University (reference number RCSTTD/403/01/17). Due to the retrospective design, the requirement for informed consent was waived by the committee in accordance with national research ethics guidelines.

All data used were deidentified before analysis to ensure participant privacy and confidentiality. No personally identifiable information (eg, names or medical record numbers) was extracted. Data were stored on password-protected computers accessible only to authorized researchers.

There was no financial compensation to participants, as this study relied on existing EMRs. No images or identifiable information about individual participants are presented in this paper. Ethical approval was obtained from the institutional review board of the Amhara Public Health Institute (APHI) (approval number: APHI/322/007). The study was conducted in accordance with the ethical principles of the Declaration of Helsinki and complied with the Ethiopian National Research Ethics Review Guideline [23].

Data quality control procedures were applied throughout the study to ensure reliability and validity. These procedures involved consistency checks and the correction of missing entries. Data entry was supervised by experienced clerks, and predefined validation rules flagged inconsistencies.

A total of 12,871 study participants were included in this study; the majority were between the ages of 38 and 47 years (n=4468, 34.7%), suggesting a largely middle-aged population; 7688 (59.7%) of the sample were female, indicating a gender imbalance; the majority were married (n= 4921, 51.2%) and had completed secondary school education (n=2953, 31.3%), indicating a moderate level of education; a sizable portion lived in urban areas (n=8878, 77.1%), suggesting greater representation of urban areas; and the vast majority were Orthodox Christians (n=8714, 91.5%), representing a highly homogeneous religious composition (see Table 1).

Among 12,871 people living with HIV at ART initiation, most exhibited preserved functional status (working: n=11,970, 93.1%) with low TB prevalence (n=219, 1.7% positive), consistent with late ART presentation in Ethiopian cohorts (see Table 2). The median baseline CD4 count was 225 (IQR 131‐367) cells/mm³.

As seen in the descriptive statistics, the prevalence of mortality prediction among people living with HIV was 20%. This shows that the dataset was imbalanced; most observations (80%) were concentrated in the majority class (ie, alive). The SMOTE oversampling strategy added 6186 synthetic observations from the minority group (ie, dead) to balance the unbalanced distribution of the outcome variable. Therefore, the class distribution for mortality prediction among people living with HIV was balanced using SMOTE, resulting in a symmetric training dataset with 8241 observations in each class (alive and dead) to support the development of reliable and robust predictive models (see Figure 3). To avoid data leakage, the dataset was first split into training (80%) and testing (20%) subsets. SMOTE oversampling was then applied only to the training data, while the test set remained in its original, imbalanced form. This ensured unbiased performance evaluation on unseen data.

ROC curves for 7 ML models trained on the original imbalanced dataset (20% mortality prevalence) demonstrated clinically realistic discrimination (AUC range: 0.70‐0.86). Gradient boosting achieved superior performance (AUC=0.859), followed by XGBoost (AUC=0.835) and logistic regression (AUC=0.824). Random forest (AUC=0.809) and Naive Bayes (AUC=0.803) showed moderate discrimination, while k-nearest neighbor (KNN) and decision tree showed lower discrimination compared to ensemble models (see Figure 4).

SMOTE-balanced models (training only, evaluated on the original imbalanced test set) demonstrated stable discrimination with modest recall improvements but precision trade-offs typical of oversampling. Logistic regression achieved AUC=0.813 (gain +0.0 from original), decision tree AUC=0.700 (−0.006), and KNN AUC=0.705 (−0.009). Tree-based ensembles maintained robustness: andom forest AUC=0.794 (−0.015), gradient boosting AUC=0.837 (−0.022), XGBoost AUC=0.819 (−0.016), and Naive Bayes AUC=0.797 (−0.006). Original configurations outperformed SMOTE variants (median ΔAUC=−0.013 [IQR -0.019 to 0.008]), favoring models trained on unaltered data for potential future clinical implementation, pending external validation (see Figure 5).

Model performance comparison across original and SMOTE-balanced configurations demonstrates gradient boosting original’s superiority (F₁-score=0.619, AUC=0.859), outperforming XGBoost original (F₁-score=0.609, AUC=0.835). SMOTE-trained models showed recall gains but precision losses: XGBoost SMOTE (F₁-score=0.592, AUC=0.819; ΔF₁-score=−0.017), random forest SMOTE (F₁-score=0.576, AUC =0.794; ΔF₁-score=−0.021), and gradient boosting SMOTE (F₁-score=0.608, AUC=0.837; ΔF₁-score=−0.011). Ensemble methods maintained robust discrimination after balancing (AUC>0.79), whereas simpler models showed greater variability. KNN and decision tree exhibited the lowest performance across configurations, reaffirming ensemble learning’s advantage for high-dimensional, imbalanced HIV mortality prediction.

Original data configurations consistently outperformed SMOTE variants (median ΔF₁-score=−0.011, median ΔAUC=−0.013), supporting potential future clinical implementation pending external validation without synthetic oversampling. This finding aligns with real-world prevalence modeling priorities where precision preservation exceeds recall optimization for resource allocation in ART programs (see Figure 6).

The predictive performance of 7 ML models was comprehensively evaluated using accuracy, precision, recall, F₁-score (primary), and AUC on an 80:20 stratified train-test split. A critical distinction is that all preprocessing procedures, including SMOTE, were applied to training data only; the final evaluation used the original imbalanced test set (20% mortality prevalence) to reflect real-world deployment conditions.

On the original imbalanced test data, gradient boosting demonstrated superior balanced performance (accuracy=87.0%, precision=74.5%, recall=52.9%, F₁-score=0.619, AUC=0.859), outperforming XGBoost (F₁-score=0.609, AUC=0.835) and random forest (F₁-score=0.597, AUC=0.809). Ensemble methods consistently exceeded single classifiers, followed by Naive Bayes (F₁-score=0.562) and logistic regression (F₁-score=0.553).

SMOTE sensitivity analysis (training only) yielded recall gains but precision losses: XGBoost recall improved by +8.2% (F₁-score=0.592, ΔF₁-score=−0.017) and gradient boosting by +14.2% (F₁-score=0.608, ΔF₁-score=−0.011). Original configurations outperformed SMOTE variants across top models (median ΔF₁-score=−0.011 [IQR 0.006-0.019], ΔAUC=−0.013 [IQR 0.008-0.019]), confirming unaltered EMR data superiority for clinical precision.

Gradient boosting (original) was selected as optimal due to the highest F₁-score, cross-validation stability, and SHAP interpretability, establishing moderate-to-strong discrimination performance for 20% prevalence mortality prediction in resource-limited settings (see Table 3).

Association rule mining using the Apriori algorithm (mlxtend; rules with support ≥0.065, confidence ≥0.64, and lift >4.6 were retained for interpretation) identified high-risk sociodemographic profiles among baseline characteristics. The dominant pattern—“rural residence + age 38‐47 years + no formal education + low baseline CD4 → mortality”—exhibited support=0.0686 (6.9% prevalence), confidence=68.3%, and lift=4.76, indicating that this subgroup showed a 4.76-fold higher co-occurrence with mortality compared with the baseline mortality prevalence (20% baseline prevalence).

In addition, demographic context (Christian religion, TB-negative status) maintained high lift values (4.62‐4.74), indicating that this subgroup exhibited the strongest co-occurrence pattern with mortality in the unadjusted association analysis (see Table 4). These association rules reflect co-occurrence patterns and should not be interpreted as causal risk factors.

The gradient boosting model demonstrates strong and clinically plausible classification performance (see Figure 7), which presents the confusion matrix of the optimized gradient boosting classifier.

Gradient boosting trained on the original dataset was selected as the optimal model, and the SHAP summary analysis was identified as the most influential baseline predictors of mortality among 12,871 people living with HIV at ART initiation. The educational level was ranked as the strongest predictor, with lower education associated with positive SHAP values (increased mortality risk) and higher education demonstrating consistent protective effects. Residence was the second most important factor, with rural residence contributing to higher predicted mortality and urban residence showing protective effects. The baseline CD4 count exhibited a clear biological gradient: lower CD4 values clustered on the positive SHAP side, indicating increased mortality risk, whereas higher CD4 counts were associated with negative SHAP values, reflecting protection. Marital status and age showed moderate influence, with older age generally increasing predicted risk. Functional status also contributed meaningfully, as ambulatory or bedridden states increased mortality risk compared to working status. In contrast, sex, religion, and TB screening results demonstrated relatively small SHAP magnitudes, indicating limited impact on overall prediction. Overall, the SHAP ranking confirms that socioeconomic vulnerability and immunological status at ART initiation are the dominant determinants of predicted mortality risk (see Figure 8).

The SHAP waterfall plot illustrates the individual prediction for a 45-year-old patient. The model output is expressed on the log-odds scale for the mortality class (dead=1), with the cohort baseline expected value of E[f(X)]=−1.886, representing the average predicted mortality risk. For this patient, the cumulative contribution of key baseline features shifts the model output to f(x)=−0.653. Because this value is substantially higher than the cohort baseline, it indicates an elevated predicted mortality risk relative to the average patient, although the absolute probability remains below 0.5.

Positive SHAP values (red bars) indicate features that increase predicted mortality risk, whereas negative SHAP values (blue bars) indicate protective effects. The strongest contributor to increased mortality risk was a lack of formal education (+0.84), followed by a low baseline CD4 count of 140 cells/mm³ (+0.54). Married as marital status (+0.30) and male as sex (+0.03) provided additional, smaller risk contributions. In contrast, urban residence (−0.35) and working functional status (−0.12) exerted protective effects by reducing the predicted mortality risk. Age (45 y; −0.02), religion (+0.02), and TB screening results (~0) had minimal influence on the individual prediction (see Figure 9).

To our knowledge, this represents one of the first Ethiopia-based applications of interpretable ensemble ML (gradient boosting combined with SHAP) using baseline EMR data to predict mortality among 12,871 people living with HIV, achieving moderate but clinically realistic performance (F₁-score=0.619, AUC=0.859). Unlike earlier exploratory analyses, the final model was developed strictly using predictors recorded at ART initiation, eliminating postbaseline variables to avoid look-ahead bias and ensure valid prognostic modeling. This study contributes new evidence by demonstrating the feasibility of applying XAI models to routine EMR data to support proactive risk stratification and clinical decision-making in resource-limited settings. However, the model should be interpreted as a decision-support tool rather than a standalone clinical triage system, given its moderate sensitivity. Consistent with findings from studies published in JMIR journals, our results demonstrate the feasibility of integrating XAI models into routine clinical care to support proactive risk stratification and decision-making, particularly in low-resource settings where the efficient use of existing EMR data is critical [1-3].

Gradient boosting trained on verified baseline-only predictors achieved the best overall balance between discrimination and generalization performance (F₁-score=0.619, AUC=0.859), outperforming logistic regression, Naive Bayes, KNN, and decision tree models. While XGBoost and random forest also demonstrated competitive performance, the corrected baseline-only gradient boosting model provided the most stable and clinically interpretable results for the 20% mortality prevalence setting.

SHAP values were calculated with respect to the mortality class (dead=1). Accordingly, positive SHAP values indicate an increase in predicted mortality risk, whereas negative SHAP values represent a reduction in predicted mortality risk (protective effect).

The most influential baseline predictors included educational level, baseline CD4 count, residence, marital status, and functional status. In the illustrated case, lack of formal education and a low baseline CD4 count (140 cells/mm³) exerted strong positive contributions, substantially increasing the predicted mortality risk. Urban residence and working functional status demonstrated protective contributions.

Importantly, the baseline CD4 count was modeled as a continuous predictor in the ML algorithms, and descriptive statistics have been revised to report a median of 225 (IQR 131-367) cells/mm³ to ensure methodological consistency. Unlike preliminary analyses that yielded inflated performance estimates, the corrected baseline-only model produced moderate but clinically realistic discrimination (AUC=0.859), consistent with previously published EMR-based mortality prediction studies [4-7].

These results are consistent in accuracy with studies conducted in Nigeria and Thailand, where ensemble models achieved high predictive accuracy in the clinical context [20,26]. Several studies have explored the use of ML algorithms for mortality prediction among people living with HIV. Similarly, a study from China utilizing a support vector machine reported an accuracy of 86%, which aligns with this study’s pre-SMOTE support vector machine performance [16].

The SHAP analysis provided valuable insights into the factors influencing mortality prediction.

The SHAP global importance analysis identified the educational level as the most influential baseline predictor of mortality, followed by residence, baseline CD4 count, marital status, age, and functional status. Specifically, a lack of formal education and lower baseline CD4 values were associated with an increased predicted mortality risk, whereas urban residence and working functional status were protective.

This hierarchy reflects the combined influence of socioeconomic vulnerability and biological immunosuppression at ART initiation, underscoring that both structural and clinical determinants contribute substantially to mortality risk prediction. For instance, a study conducted in Kenya found that patients on ART for extended periods had significantly lower mortality rates, reinforcing the critical role of early and consistent treatment adherence in improving survival [27].

Ensemble methods, such as XGBoost, random forest, and gradient boosting, consistently outperformed simpler models, such as logistic regression and Naive Bayes. Similar findings were reported in studies from Northern Thailand, East Africa, and Nigeria, where ensemble models achieved superior performance in health prediction tasks [28,29].

The model comparison plot demonstrated that gradient boosting achieved an accuracy of 87.0%, precision of 74.5%, recall of 52.9%, F₁-score of 0.619, and AUC of 0.859. Although discrimination was strong, the recall of 52.9% indicates that nearly half of the mortality cases were not identified at the default classification threshold. Therefore, while the model demonstrates predictive capability, its sensitivity limits immediate deployment as a high-stakes early-warning system without threshold optimization.

This study used retrospective secondary EMR data, which may contain incomplete or inaccurately recorded variables, introducing potential information bias. Only 1 baseline record per patient was retained to avoid correlated observations, limiting longitudinal analysis. Patients who transferred to other facilities were excluded, potentially introducing selection bias.

Although the cohort required documented baseline ART initiation records, limited variable-level missingness (<10%) was addressed using median (continuous variables) and mode (categorical variables) imputation. Therefore, this was not a strict complete-case analysis. Excluding individuals on ART for less than 6 months may have introduced survivorship bias, potentially inflating performance metrics and limiting generalizability to newly initiated ART populations.

SMOTE-based oversampling may not fully replicate real-world population distributions. External validation using independent datasets was not performed, and model performance may vary in other geographic or clinical contexts. Data were drawn from a single city administration, limiting external validity.

Finally, given the moderate sensitivity (52.9%, 272/514), the model may miss a substantial proportion of high-risk patients at the default threshold. Future implementations should consider threshold calibration based on clinical priorities, particularly if minimizing false negatives is paramount.

Future studies should use multicenter or national-level data to enhance generalizability and external validity. Prospective validation studies are needed to evaluate real-world performance. Threshold optimization strategies should be explored to improve sensitivity where early mortality detection is prioritized. Integrating temporal modeling approaches may capture longitudinal treatment dynamics. Additionally, embedding explainable models within clinical workflows could support structured risk assessment, provided that implementation is accompanied by clinician oversight and contextual calibration.

This study demonstrates that interpretable ensemble ML models trained exclusively on baseline EMR data achieved moderate yet clinically meaningful discrimination in predicting mortality risk among people living with HIV initiating ART. SHAP-based interpretation confirmed that educational attainment and baseline CD4 count were the most influential predictors, followed by residence, age, and functional status, highlighting the combined impact of socioeconomic vulnerability and immunological status at treatment initiation. These findings suggest that transparent and explainable ML approaches may support early risk stratification and targeted intervention planning within HIV care programs. Nevertheless, external validation and prospective evaluation are required before integration into routine clinical decision-making.