This study retrospectively collects data from the ED of a tertiary teaching hospital located in Tainan City, Taiwan, between January 1, 2016, and December 31, 2022. The hospital comprises 783 general ward beds and 136 intensive care unit beds, and on average, receives approximately 100,000 ED visits annually. Adult patients presenting with chest pain and triaged as level 2 (urgent) were included [12]. Inclusion required complete triage data, medical history, examination results, and documented disposition. Patients were categorized into 3 groups: group GA (ACS), group GB1 (non-ACS but requiring acute care or hospitalization), and group GB2 (noncritical, discharged after short observation). Visits with missing or indeterminate data were all excluded, which was the only exclusion criterion. As such, all included visits have no missing data. The dataset was divided chronologically: records from January 1, 2016, to December 31, 2020, were used for training and from January 1, 2021, to December 31, 2022, for testing (75/25 split). Repeat visits were all included unless with missing or indeterminate data.

To maintain the clinical integrity and analytical precision required for this pilot research, we prioritized our institutional dataset over open ED datasets, such as Medical Information Mart for Intensive Care IV Emergency Department and Multimodal Clinical Monitoring in the Emergency Department.

In total, 27 variables were used for data analysis and model development. These included 3 response variables, 2 demographic variables (age and sex), and 22 clinical variables. The clinical variables consisted of 6 vital signs, 4 blood test results, and history of 12 diseases and medical conditions. Among the 27 variables, the response, sex, and medical history were categorical, while the remaining variables were quantitative (Textbox 1).

Quantitative variables were used in their raw form without prior scaling, standardization, or log-transformation. Instead, potential disparities in numerical scales were (partially) mitigated via batch normalization integrated directly within the artificial neural network (ANN) architectures. Because batch normalization primarily addresses internal covariate shift rather than correcting highly skewed raw input distributions (such as hs-TnT), a sensitivity analysis was conducted by using instead the “log-transformed hs-TnT” as an alternative input.

In alignment with Good Machine Learning Practice principles [13] regarding the clinical workflow context, we defined a precise index time anchor to prevent data leakage. The model’s prediction shall be executed at the exact moment the first laboratory panel (including hs-TnT, creatinine, and complete blood count) is reported, which typically occurs within 60 minutes of patient arrival. Only features available at or before this timestamp were used as model inputs. By anchoring the prediction at this specific decision window—after initial laboratory results are available but before final disposition or serial testing is initiated—we ensure the model functions as a true clinical decision support (CDS) tool within the intended use environment.

Patients diagnosed with ACS were classified as GA. Those without ACS were further categorized as GB1 if they had other serious conditions requiring hospitalization, transfer, or died in the ED, and as GB2 if they were discharged after ED care.

To ensure the safety and validity of the ground truth labels, we established a rigorous adjudication process for ACS, specifically focusing on type 1 myocardial infarction (MI) in accordance with the Fourth Universal Definition of MI [14]. To maintain high specificity and provide a fit-for-purpose reference standard, cases of unstable angina and type 2 MI were intentionally excluded from the target outcome group GA.

The diagnostic adjudication was a multidisciplinary effort. Initial clinical assessments were performed by board-certified emergency physicians, with definitive diagnosis and type 1 versus type 2 MI distinction determined by consulting cardiologists based on clinical presentation, serial electrocardiogram (ECG) changes, and biomarker trajectories. To further minimize label noise and information bias, we implemented a dual-referencing verification strategy: a visit was only labeled as GA if the corresponding International Classification of Diseases, 10th Revision (ICD-10 codes; I21.0-I21.4, I22.x) were documented in both the initial ED record and the final hospital discharge summary [15]. This methodology ensures that the model learns from confirmed ischemic events while excluding initial ED misdiagnoses or secondary events occurring after the index visit.

ANNs were developed in Python to predict patients’ categorical outcomes (GA, GB1, or GB2) [16]. Three submodels—ANN-GA, ANN-GB1, and ANN-GB2—were trained to estimate the probability of each respective group. Their outputs were integrated via a multiclassifier for final prediction. All submodels shared the same architecture, with training parameters tailored to their respective targets; therefore, only the development of ANN-GA is detailed below. The first, second, and third response variables were used to train ANN-GA, ANN-GB1, and ANN-GB2, respectively. To address the imbalance among GA, GB1, and GB2 cohorts, the approach of assigning different weights to different patient groups, or down-sampling the dominating cohort, was adopted when training our models. We did not apply the synthetic minority oversampling technique. We used the scikit-learn (0.23.2) and the tensorflow (2.8.0) libraries for training our models.

The ANN-GA model was developed following a procedure similar to our previous work [17]. A 5-fold cross-validation protocol [18] was applied to the training dataset, dividing it into 5 equal subsets. In the ith iteration (i=1, 2, ... , 5), the ith subset was held out for internal validation, while a model M_i(x) of the form:

(where x represents the input vector of 24 decision variables) was trained on the remaining 4 subsets. Central to the model M_i(x) is the nonlinear function f_i(x), which was obtained by training a fully connected ANN consisting of 4 layers. Each hidden layer (except the last) comprises batch normalization [19], affine transformation, and nonlinear activation. The batch normalization recenters and scales the input to stabilize and accelerate training, the affine transformation applies weights and biases to the input, and the activation introduces nonlinearity. This represents a standard layer structure of the ANNs [16]. The final layer of the network omitted a nonlinear activation, instead passing its output to a standard sigmoid function to generate the final outcome. The input dimensions of layers 1 to 4 were 24, m₂, m₃, and m₄, while the output dimensions were m₂, m₃, m₄, and 1, where m₂, m₃, and m₄ are network design parameters that were set between 5 and 50. The first 3 layers used hyperbolic tangent activations. Each model had fewer than 10,000 trainable parameters. These parameters were optimized based on minimizing the binary cross-entropy function, and the Adam optimizer [20] with recommended settings was used for the optimization procedure. Finally, the ANN-GA model, M_GA(x), was taken as the average of the 5 models; that is, MGA(x):=15∑i=15Mi(x). As the values of M_GA(x) are always between 0 and 1, the output of M_GA(x) is viewed as the predicted probability of belonging to group GA.

In the next step, the outputs of the 3 uncalibrated ANN models were jointly normalized so that their predicted probabilities for a given event summed to unity. Each model then underwent separate calibration using isotonic regression [21] to align the individual binary predictions with observed event rates. Because this independent calibration generally breaks the sum-to-unity property, a final softmax function with optimized temperature scaling was applied to the joint postisotonic outputs to restore coherent multiclass probabilities. Specifically, let p_A, p_B1, and p_B2 be the postisotonic probabilities for GA, GB1, and GB2 classification, respectively. In the final step, these probabilities are modified as follows:

where T denotes the temperature scaling factor, which is optimized to minimize the binary cross-entropy loss of the recalibrated probabilities. The optimized temperature scaling factor T is 0.935. Figure 1A and B illustrate the layer structure and the flow for constructing the ANN models, including the 2-step calibration pipeline. The hyperparameter configurations used for model training are summarized in Table 1.

To verify the similarity between training and testing datasets, we compared feature distributions by calculating category percentages for categorical variables and means with SDs for quantitative variables. Performance evaluation was conducted based on the 2-step calibrated probabilities obtained by applying (calibrated) models to the testing set, with the area under the receiver operating characteristic curve (AUROC) serving as the primary metric. Optimal classification thresholds were derived from the training set’s receiver operating characteristic (ROC) curves by maximizing the Youden index calculated from the calibrated probabilities [22]. These thresholds were then applied to evaluate model sensitivity (recall), specificity, positive predictive value (PPV, or precision), negative predictive value (NPV), and F₁-score. Precision-recall (PR) curves and the areas under the precision-recall curves (AUPRC) were also analyzed. Finally, to evaluate clinical utility, decision curve analysis was applied to the 2-step calibrated ANN models, benchmarking their performance against traditional “treat-all” and “treat-none” strategies.

Furthermore, to evaluate whether the ANN architecture offers superior predictive power, we benchmarked the ANN models against their logistic regression (LR) counterparts (LR-GA, LR-GB1, and LR-GB2). Note that by restricting function f_i(x) to be affine, the model M_i(x) becomes an LR model. To ensure a rigorous comparison, the LR models were constructed as ANN models with such a specific architectural constraint, optimized, and calibrated via the same computational process. As their ANN counterparts, the LR models were trained using the same training data by optimizing the same loss function, following the same data imbalance strategies, and using the same hyperparameters for the optimization engine. Furthermore, they were tested on the identical testing datasets and evaluated using the same performance metrics as the ANN models.

Moreover, in order to assess potential performance disparity between different patient groups, subgroup analysis with respect to age, sex, and kidney function was performed. The kidney function was evaluated based on estimated glomerular filtration rate (eGFR), which was calculated using the 2021 Chronic Kidney Disease Epidemiology Collaboration formula [23]. The model’s performance on different subgroups was compared based on their ROC curves and the corresponding AUROCs.

Multiclassifiers were constructed for the final prediction using the 2-step calibrated outputs of the 3 ANN models. Each model output was first divided by the respective optimal threshold for binary classification, yielding the scores o_GA, o_GB1, and o_GB2 for each patient. By dividing the respective optimal threshold, the scores o_GA, o_GB1, and o_GB2 measure a patient’s “distance” to the clinically accepted risk requirement for each classification. Three classification rules were evaluated for making the final classification prediction:

Figure 1C illustrates the multiclassification process. Each multiclassifier was applied to the testing data, and their performance was evaluated by comparing miss rates, sensitivity, specificity, PPV, NPV, and F₁-score.

Finally, we benchmark a multiclassifier based on ANN models with Youden optimal thresholds against that based on the LR models to evaluate the performance gains afforded by the ANN architecture. We further conducted sensitivity analyses by varying the GA-classification threshold, allowing us to explore the inherent trade-off between identifying high-risk ACS (GA) and low-risk (GB2) cases.

To evaluate the potential of ANN models in reducing ED overcrowding, we analyzed GB2 patient records from the testing dataset, focusing on ED LoS and the number of hs-TnT tests received. While 2 tests are typically required before discharge, some patients were released after 1 test. By identifying such cases, we estimated the average time saved if GB2 patients were discharged earlier based on model predictions, thereby assessing the model’s impact on improving patient flow.

To evaluate the contribution of each feature variable to the predictive power of the models, Gini diversity indices [24] were calculated for classifying patients into GA, GB1, and GB2. Variables with notably high importance were selected, and ANN models were trained and calibrated using only these variables or the remaining ones. The performance of these models was compared to that of the full-variable models to assess whether key features alone provided sufficient predictive power. Furthermore, the Shapley Additive Explanations (SHAP) value analysis [25] was conducted on the ANN models to clarify the impact of each feature variable on the model’s output. The results were examined to seek a clinical explanation of the model’s predictive power.

Statistical analyses were conducted with SPSS for Windows (version 24.0; IBM Corp). Categorical variables were compared with the chi-square test, and continuous variables (means and SD) with the independent 2-tailed t test.

A formal study protocol was not prospectively prepared for this retrospective analysis. Additionally, the study was not registered in a public clinical trial database, as it used existing electronic health records for model development and evaluation. The study was approved by the institutional review board of the National Cheng Kung University Hospital, Tainan, Taiwan (A-ER-111‐199). The study was conducted following the principles set forth in the Declaration of Helsinki. The requirement for informed consent was waived due to the retrospective nature of the study. Study data were deidentified before processing to protect the privacy of the participants. Patients or the public were not involved in the design, conduct, reporting, or dissemination plans of this research. This study was reported according to the TRIPOD+AI (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis Using Artificial Intelligence) guidelines, and the completed checklist is provided in Checklist 1.

Figure 3 presents the ROC and PR curves of the 3 ANN and the 3 LR models for both training and testing datasets. The full confusion matrices and class-wise performance metrics, including AUROC, accuracy, sensitivity, specificity, PPV, NPV, F₁-score, and AUPRC at respective optimal cutoff thresholds, are summarized in Tables 2 and 3. For multiclass classification, full confusion matrices and performance metrics under 3 decision mechanisms (S1-S3) are summarized in Tables 4 and 5. The ANN-based multiclassifiers (thereafter referred to as ANN-S1, ANN-S2, and ANN-S3) were able to achieve high sensitivity in identifying GA patients and high PPV and high sensitivity in identifying GB2 patients. Notably, ANN-S3 yielded 0.941 sensitivity (95% CI 0.908‐0.973) for GA and 0.837 sensitivity (95% CI 0.824‐0.850) and 0.911 PPV (95% CI 0.900‐0.921) for GB2.

Furthermore, Tables 6 and 7 provide the complete confusion matrices and performance metrics for multiclassifiers using the S3 mechanism. These summaries compare the multiclassifiers based on ANN and LR models (referred to as ANN-S3 and LR-S3, respectively)—both using Youden-optimal thresholds—alongside an ANN variant with a reduced GA classification threshold (referred to as ANN-S3-L). For ANN-S3-L, the GA classification threshold was set at 0.0167 (significantly lower compared to 0.0744 for ANN-S3 and 0.0553 for LR-S3). This value was determined by identifying the maximum threshold at which the ANN-GA model achieved a sensitivity exceeding 0.95 on the training data, a conservative approach intended to prioritize clinical safety and minimize false negatives.

Calibration curves with Brier scores (95% CIs) for ANN-GA, ANN-GB1, ANN-GB2 models on the training and testing datasets are presented in Figure 4. The results indicate that the calibrated models render predicted probabilities more aligning with the observed risk. ANN-GB2 appears to be very well calibrated on both datasets, while ANN-GB1 somewhat underestimates and ANN-GA overestimates the risk. The results of the decision curve analysis are illustrated in Figure 5. The decision curves show that across a broad range of probability thresholds, all models demonstrated superior clinical utility compared to the traditional “treat-all” and “treat-none” baseline strategies. Notably, for the GB1 and GB2 models, this improved net benefit was maintained across almost the full range of probability thresholds.

Analysis of subgroup performance was based on the ROC curves of the calibrated ANN models on different subgroups (testing) data. Specifically, we consider 3 age bands (below 65, 65‐85, and above 85 years), male and female sexes, and 4 eGFR strata (below 30, 30‐60, 60‐90, and above 90). The results are presented in Figure 6. Figure 6D-F shows that the ROC curves for male and female subgroups closely align with those of the full dataset, indicating negligible disparity of the model’s predictive power among different sex subgroups. However, notable disparities exist regarding age and eGFR. Specifically, the model underperformed in older-age and low-eGFR subgroups, which exhibited both lower AUROC scores and ROC curves that fell substantially below the aggregate baseline.

To ensure that the 142 patients overlap between the adjacent years of 2020 (belonging to the training cohort) and 2021 (belonging to the testing cohort) did not bias our findings via static baseline variables (eg, age, sex, and comorbidities), a sensitivity analysis was performed by excluding the 151 testing visits by these individuals. The resulting performance metrics remained tightly aligned with the primary analysis. In particular, the AUROC, sensitivity, and PPV for both GA and GB2 classifications remained well within the original 95% CIs. This confirms that cohort overlap did not artificially inflate model performance. A comprehensive comparison is detailed in Tables S7 and S8 in Multimedia Appendix 1.

Furthermore, to evaluate whether the heavy positive skew of the raw hs-TnT data impacted model performance, we performed a sensitivity analysis by retraining and calibrating the ANN models under identical conditions and replacing the raw hs-TnT values with their log-transformed counterparts. Models trained with log-transformed hs-TnT showed slightly lower performance; yet, their results remained tightly aligned with our primary architecture, as reflected by heavily overlapping 95% CIs. Specifically, the training AUROC scores for the primary vs log-transformed (“-T”) models were ANN-GA: 0.957 (95% CI 0.948‐0.965) vs ANN-GA-T: 0.951 (95% CI 0.941‐0.960), ANN-GB1: 0.809 (95% CI 0.799‐0.819) vs ANN-GB1-T: 0.794 (95% CI 0.784‐0.804), and ANN-GB2: 0.876 (95% CI 0.870‐0.881) vs ANN-GB2-T: 0.869 (95% CI 0.864‐0.875). This demonstrates that while batch normalization primarily stabilizes mini-batch activations, the neural network’s architecture was robust to the empirical distribution and skewness of the raw cardiac biomarker data. A comprehensive performance comparison, including AUPRC scores, is provided in Table S9 in Multimedia Appendix 1.

In the testing dataset, 3237 patients were categorized as GB2, indicating noncritical conditions. Among them, 3087 had complete data for ED length-of-stay analysis. Of these patients, 1247 received a single hs-TnT test (T1 subgroup, consistent with a single-sample rule-out strategy [26]; median ED LoS 139, IQR 108-195 minutes), whereas 1840 received 2 hs-TnT tests (T2 subgroup, consistent with an every-3-hour serial testing strategy [27]; median ED LoS 285, IQR 251-366 minutes). Based on the clinical variables available at the time of the initial laboratory panel, the ANN-S3 multiclassifier identified 2653 patients as GB2, while 434 GB2 patients were classified as non-GB2.

Among the model-identified GB2 patients, 1170 were from the T1 subgroup, representing 93.83% (1170/1247) of patients who were discharged after a single hs-TnT test in actual clinical practice. This high concordance suggests that the model frequently aligned with physicians’ real-world decisions to manage selected low-risk patients using a single-sample strategy. In addition, 1483 model-identified GB2 patients were from the T2 subgroup, representing 80.60% (1483/1840) of patients who underwent serial hs-TnT testing. These patients represent a retrospectively identified subgroup who might have been considered eligible for a single-sample protocol based on the risk-assessment model, although they were managed with serial testing in actual practice.

The longer median LoS observed in the T2 subgroup compared with the T1 subgroup suggests that serial hs-TnT testing was associated with longer ED stays in this retrospective cohort. However, these observational differences should not be interpreted as direct evidence that model implementation would reduce LoS by the same magnitude, because the original decision to perform single versus serial testing was likely influenced by physician-perceived risk and other clinical workflow factors. Therefore, our findings, as illustrated in Figure 7, should be interpreted as demonstrating the model’s potential to enhance ED throughput by identifying additional low-risk candidates for a single-sample strategy, rather than as definitive evidence of actual operational improvement. The actual impact on ED LoS, discharge timing, and patient flow will require prospective evaluation in real-world clinical workflows.

For each group classification, the 7 most important feature variables (referred to as “key feature variables”) identified by Gini diversity analysis are as follows:

The ANN models trained using only key feature variables were referred to as ANN-GA-RED, ANN-GB1-RED, and ANN-GB2-RED, respectively. Models trained using all features except the key feature variables were named ANN-GA-RMV, ANN-GB1-RMV, and ANN-GB2-RMV, respectively. Figure 8 displays the ROC curves of these models, along with those of ANN-GA, ANN-GB1, and ANN-GB2. The detailed performance metrics of these models on the testing data are summarized in Table S10 in Multimedia Appendix 1.

Regarding the impact of each feature variable on the model’s output, the results of SHAP value analysis are presented in Figure 9. For each model, the top 7 “high impact” feature variables are as follows:

Apparently, the predicted probabilities rendered by ANN-GA and ANN-GB2 models are driven by 5 common feature variables: CAD, hs-TnT, age, sex, and temperature. In addition, the output of the ANN-GB1 model is predominantly driven by the hs-TnT variable, whose impact is tripled compared to those of other variables.

In this study, machine learning models were developed to assist in identifying patients with ACS (GA), those without ACS but with other severe illness (GB1), and low-risk patients (GB2). The individual ANN classification models—ANN-GA, ANN-GB1, and ANN-GB2—all demonstrated impressive performance based on testing AUROC scores: ANN-GA and ANN-GB2 achieved scores close to or greater than 0.9, while ANN-GB1 achieved 0.824. In terms of AUPRC scores, ANN-GB1 and ANN-GB2 achieved 0.55 and 0.948, respectively, indicating a good balance between precision and recall. ANN-GA performed relatively poorly on the AUPRC score, achieving only 0.391. However, this is still a significant improvement compared to a no-skill classifier, for which the AUPRC score would be less than 0.05. The superior predictive powers of the ANN models are evidenced by their significant performance gains over their LR counterparts. This margin is clearly demonstrated by the ROC curves in Figure 3, and the comprehensive metrics are summarized in Tables 2 and 3.

The primary objectives for multiclassification are two-fold: (1) maximizing sensitivity for patients with ACS (GA) due to the life-threatening nature of the condition and (2) ensuring high sensitivity and PPV for low-risk patients (GB2) to streamline the ED patient flow. Among the models evaluated, the ANN-S3 classifier emerged as the superior choice. While ANN-S2 matched its sensitivity for GA, it experienced a lower PPV for GB2 patients. ANN-S1 achieved (almost) identical GB2 performance but exhibited notably lower sensitivity for GA cases. The LR counterpart, LR-S3, proved inferior due to prohibitively low GA and GB2 sensitivities (0.808 and 0.672, respectively). To prioritize clinical safety, the ANN-S3-L variant was developed by lowering the GA classification threshold from 0.0744 to 0.0167. This adjustment resulted in a much higher GA sensitivity (0.966) and NPV (0.998), though it reduced GA specificity from 0.902 to 0.797, as well as GB2 sensitivity from 0.837 to 0.787. This shift ultimately diminished the model’s ability to identify low-risk patients, highlighting the inherent trade-off between clinical safety and operational efficiency.

The S3 multiclassifier uses adjusted scores to differentiate between GB1 (requiring acute care) and GB2 (suitable for discharge after observation) classifications. To prioritize patient safety, the GB1 adjustment factor is nearly 3 times larger than that of GB2 (1/0.2441 vs 1/0.7069). This scaling biases borderline decisions against the GB2 classification, favoring clinical caution over statistical parity. These adjusted scores essentially measure a patient’s “distance” from the clinically accepted risk requirement for respective classification. For example, if a patient’s risk profile exceeds the GB1 threshold (adjusted score>1) but remains below the GB2 threshold (adjusted score<1), the mechanism defaults to GB1. This design is based on the principle that meeting a specific “required” risk level is a more significant clinical milestone than achieving a high raw score that fails to cross a safety threshold. While we acknowledge that this lack of statistical parity may reduce GB2 sensitivity, we view this as a necessary trade-off to enhance the safety of high-risk patients.

Notably, the ANN-based multiclassifier S3 misclassified 5 ACS (GA) cases as “low-risk” (GB2), as shown in Table 6. We have conducted a detailed review of the 5 ACS cases to identify specific failure modes. The misclassified cohort shared a specific “low-risk” profile: (1) a lack of prior CAD history, (2) a relatively young age (median<65 years), (3) hemodynamic and respiratory stability, and (4) initial hs-TnT levels below the 99th percentile URL of 14 ng/L (with 6 of 7 cases<11 ng/L). Identifying ACS in patients who lack traditional risk factors, present as physiologically stable, and maintain “normal” biomarkers represents a significant diagnostic boundary for the model. Such cases remain a formidable challenge even for experienced clinicians [26]. Recognizing these failure patterns is vital for the responsible integration of AI in the ED, ensuring that the model serves as an objective support tool while physician judgment remains the primary safeguard for atypical or early presentations.

Regarding subgroup performance, our analysis revealed that model predictive power decreases in older populations and in patients with lower eGFRs. These results reflect the inherent complexity of diagnosing acute conditions among the older patients in the ED, where they often present with atypical symptoms and multiple comorbidities that obscure traditional clinical signals [27,28]. The observed performance drop suggests that ACS-associated multivariable patterns are significantly more heterogeneous in older cohorts than in younger ones. Moreover, while hs-TnT was a high-impact variable across all classification groups, renal impairment acts as a known confounder. Chronically elevated baseline hs-TnT levels in patients with renal disease complicate the distinction between acute myocardial injury and chronic cardiac strain [29]. Consequently, the diagnostic utility of hs-TnT diminishes as eGFR declines, leading to the observed degradation in model performance. These findings underscore that a “one-size-fits-all” model may be insufficient for high-risk, complex populations. To ensure the responsible deployment of AI in clinical settings, our future work will focus on developing specialized models tailored to the unique physiological and biomarker profiles of older patients and patients who are renally impaired.

Furthermore, the observed performance disparities may stem from the underrepresentation of specific subgroups during training. For instance, patients older than 85 years of age constitute only 7.8% of the training dataset, while those with an eGFR of 30‐60 mL/minute/1.73 m² and below 30 mL/minute/1.73 m² represent just 13% and 11%, respectively. To mitigate this training bias in future iterations, oversampling strategies such as the synthetic minority oversampling technique could be used to better balance these cohorts.

The current 2021 American Heart Association (AHA) and American College of Cardiology (ACC) and 2023 European Society of Cardiology guidelines emphasize serial high-sensitivity troponin testing as the primary standard of care for the evaluation of ACP. Our findings are not intended to supersede these class I recommendations; rather, they explore the potential of AI to support existing “single-sample rule-out” concepts for specific low-risk populations, as discussed in the 2021 AHA and ACC guidelines for patients with prolonged symptom onset. By integrating hs-TnT with 23 other demographic and clinical feature variables, our ANN model offers a multidimensional risk assessment that may help clinicians interpret “gray-zone” troponin levels in older or comorbid patients. This approach serves as a CDS tool, providing an additional layer of evidence to assist physicians in their disposition decisions while remaining within the safety boundaries established by contemporary clinical practice.

While clinical guidelines such as the European Society of Cardiology 0/1 hour or 0/2 hour algorithms are established standards for ruling out ACS, their impact on operational efficiency in real-world practice is often limited. Findings from the European Society of Cardiology 0h/1h Troponin Rule-Out Protocol trial suggest that implementing the 0/1 hour protocol may not significantly reduce ED LoS or increase early discharge rates compared to standard care [30]. This discrepancy exists because actual clinical workflows are fundamentally driven by physician judgment, which synthesizes patient history and multifaceted clinical assessments rather than relying solely on serial laboratory values. Our ANN model addresses this gap by integrating 24 decision variables, including renal function (creatinine), hematological parameters (WBC and platelet counts), vital signs, and extensive medical history. By using this multidimensional approach, the model identifies low-risk features more comprehensively than simple heuristics—such as isolated troponin thresholds—and more closely mirrors the integrative nature of expert clinical reasoning. Furthermore, the objective risk stratification provided by the AI model serves as a reliable “second opinion.” This decision support is particularly valuable in high-pressure ED environments, where it can mitigate decision fatigue and cognitive burden among emergency physicians. By enhancing the confidence of clinicians in early discharge decisions, the model has the potential to optimize patient flow and alleviate ED overcrowding without compromising the established safety standards of current practice.

Our findings suggest that while point-of-care testing is invaluable for rapid, lean evaluations, an AI-augmented strategy may provide a distinct, complementary pathway for deeper risk stratification. Rather than imposing a “one-size-fits-all” mandate, this model offers an objective decision-support tool that reinforces the physician’s initial clinical suspicion. For patients already requiring a standard laboratory workup, the model leverages 24 variables to facilitate a confident, single-sample disposition, which may achieve a substantial reduction in ED LoS compared to traditional serial testing protocols. In our retrospective cohort, patients managed with a single-sample strategy had a median LoS of 139 (IQR 108-195) minutes, compared with 285 (IQR 251-366) minutes among those who underwent serial testing. The difference suggests the potential operational relevance of identifying additional candidates for a single-sample protocol. One should not, however, interpret this difference as a predicted reduction in LoS after model implementation, as the comparison is observational and subject to confounding by indication. Prospective evaluation is required to determine the actual operational impact of a model-driven decision support tool on ED throughput.

The predictive power of each ANN model was primarily driven by a set of key feature variables, as demonstrated by the performance variance in models trained with, with only, and without these variables. These essential features—detailed in the Importance of Feature Variables and Model Interpretability section—encompass patient age, triage blood pressure, prior history of CAD, WBC count, platelet count, serum creatinine, and hs-TnT. The inclusion of age, hypertension, and prior CAD history aligns with traditional cardiovascular risk profiles [31-33]. Models’ reliance on serum creatinine mirrors the documented association between the severity of chronic kidney disease and an increased likelihood of cardiovascular events [34]. Moreover, the predictive weight of WBC and platelet counts reflects the systemic inflammation and platelet activation inherently linked to atherothrombotic processes often observed in patients with ACS [35]. Finally, the inclusion of hs-TnT, the cornerstone biomarker for myocardial injury, reinforces the medical relevance of our feature selection. These key features contribute the primary diagnostic weight. Notably, the “reduced” models—trained exclusively on these high-impact variables—occasionally outperformed the 24-variate “full” models in specific test settings. Furthermore, for categories GB1 and GB2, performance was slightly higher on the testing data than on the training data. This phenomenon likely results from the random fluctuations of low-contributory variables in the training set, which can lead to overfitting. By focusing on high-impact features, the reduced models achieve a form of “implicit regularization,” effectively filtering out stochastic noise. Consequently, these parsimonious models may offer superior operational robustness for real-world clinical deployment, as the marginal utility of the remaining variables is offset by the risk of overfitting to cohort-specific noise.

SHAP value analysis demonstrates high consistency and medical interpretability across both high-risk (GA) and low-risk (GB2) classifications. The most influential variables—specifically hs-TnT, age, sex, and a history of CAD—are firmly grounded in established clinical evidence [36]. In particular, the dominant role of hs-TnT as a fundamental biomarker for myocardial injury aligns perfectly with current diagnostic standards for ACS. Notably, these findings underscore that the model’s predictive power is derived from medically plausible features rather than mere statistical correlation. This alignment between model-identified variables and clinical intuition supports the causal relevance of the model, distinguishing it from conventional “black-box” approaches. Such interpretability is essential for fostering clinician trust and ensuring the responsible integration of AI into medical decision-making.

Most existing studies have focused primarily on the diagnostic performance of AI models in identifying or ruling out ACS [37]. However, relatively few have proposed robust methodologies to evaluate whether the implementation of such models in real-world clinical workflows could yield tangible benefits. In this study, we conducted a retrospective analysis using historical data to quantitatively assess the potential impact of incorporating an AI-based CDS system into the emergency management of patients presenting with ACP. In the high-risk patient subgroup, our model may facilitate earlier recognition of non-STEMI and unstable angina, thus enabling more timely intervention and potentially improving patient outcomes. For low-risk patients, our findings suggest that, compared to physicians’ original clinical decisions, the AI model could safely identify a greater proportion of patients eligible for the single-sample rule-out protocol [38]. This may reduce unnecessary serial blood testing and prolonged ED observation, thereby lowering health care costs and alleviating ED crowding [3].

Furthermore, the integration of AI models into clinical workflow for decision-making support may help mitigate cognitive burden and decision fatigue among emergency physicians, ultimately reducing the risk of burnout [39]. On the other hand, while our ANN model demonstrates high classification performance in a retrospective setting, we acknowledge that significant barriers remain before it can be integrated into routine emergency care. These include sociotechnical challenges such as clinician acceptance, workflow integration, and the medical-legal implications of deviating from serial troponin testing protocols. Our findings should be interpreted as a preliminary step in exploring AI-augmented risk stratification. Future research must focus on prospective validation and implementation science to ensure that such models provide tangible clinical benefits without compromising patient safety or increasing the cognitive load on ED staff.

To ensure the safe and responsible implementation of our model, we emphasize that the AI system is designed as a CDS tool intended to function within a human-AI team framework. Rather than replacing physician judgment, the model provides context-specific risk assessments that emergency physicians can integrate into their clinical workflow. For high-risk cases (GA and GB1), the system can actively alert clinicians, whereas for low-risk patients, it serves as a passive supportive resource. To mitigate risks such as overreliance or user error, clinicians are provided with the basis for model outputs and informed of the system’s limitations. Furthermore, in alignment with Good Machine Learning Practice principles, we acknowledge that the performance of deployed models must be subject to ongoing monitoring in “real-world” use. Our planned future work includes establishing a dedicated monitoring platform to detect dataset drift or performance degradation over time. This framework will include strict controls for retraining and model updates to manage risks such as overfitting or unintended bias, ensuring the model’s safety and effectiveness throughout its total product life cycle.

This study has several limitations. First, this study’s single-center, retrospective design inherently limits the geographic and ethnic generalizability of our findings. We acknowledge that some key baseline characteristics—such as cardiovascular risk profiles, BMI thresholds, and renal function—vary significantly across diverse populations and health care systems. Consequently, the model’s performance may reflect our specific local cohort and might not be immediately applicable to other clinical settings. External validation using larger, multicenter datasets is a mandatory prerequisite before this model can be considered for universal clinical implementation.

Second, our study used ED disposition as the primary surrogate for the “low-risk” (GB2) ground truth, which may not capture adverse events occurring outside the hospital. Although we confirmed the 30-day survival of 8556 patients through subsequent clinical records, remaining patients had no further follow-up in our ED, limiting our ability to definitively confirm their survival status. This is again a characteristic limitation of single-center retrospective designs. However, given the negligible 30-day MACE rate of 0.2%, the GB2 label remains a clinically reliable benchmark for training and evaluating our AI models.

Third, the absence of symptom-onset timing in our dataset represents a significant constraint. The 2021 AHA and ACC Chest Pain Guidelines mandate that single-sample rule-out strategies apply only to patients presenting more than 3 hours after symptoms begin. Since our dataset lacks this temporal data, the model’s clinical use should be limited to late-presenting cohorts to maintain diagnostic safety. Future research will focus on safely extending the model’s utility to early presenters while preserving a high NPV.

Fourth, a notable limitation of this study is that the model does not use raw ECG waveforms, structured ECG interpretations, or qualitative clinical descriptors (eg, specific chest pain characteristics) as direct inputs. These variables are core components of established risk stratification tools such as the History, ECG, Age, Risk factors, Troponin score, or Emergency Department Assessment of Chest Pain Score; their absence in a structured format within our retrospective dataset precluded a direct comparison with these traditional heuristics. However, because our unselected study population included all eligible ED visits regardless of ECG presentation, and since our ground-truth labels were derived from final clinical diagnoses—which inherently incorporate ECG and semiological findings—the model effectively learned to correlate its 24 clinical variables with outcomes already influenced by these factors. While our SHAP analysis highlights the importance of traditional risk factors, the ANN architecture was specifically leveraged to capture complex, multidimensional interactions across the available parameters. Currently, the model is intended to augment, rather than replace, initial physician ECG screening. Future prospective work aims to integrate digitized ECG signal analysis and granular clinical descriptors to further strengthen diagnostic safety and benchmark the model’s additive value against established clinical scores.

Fifth, this model may be subject to spectrum bias, as the inclusion criteria were limited to patients triaged as level-2 (urgent). While this prioritized a high-acuity cohort with the most immediate clinical need, it excludes patients undertriaged to lower levels who may still harbor high-risk pathology. This reliance on triage levels partly stems from a lack of granular, structured chest pain data—such as pain radiation, quality, or alleviating factors—which was primarily captured as unstructured narrative text. Although our current variables align with the standard of care, the absence of these qualitative descriptors might lead to the omission of atypical presentations. Future integration of natural language processing to extract clinical features from nursing notes could mitigate this bias, allowing for a broader patient spectrum and improved differentiation between ACS and other critical conditions across all triage categories.

Sixth, a potential selection bias exists due to the exclusion of 1435 visits with missing or indeterminate data, such as incomplete triage records, examination results, or medical histories. While our analysis shows that the final cohort of 17,935 visits remains highly representative of our institution’s urgent chest pain population, these exclusions may slightly impact the generalizability of our findings. The potential bias can be mitigated when the models are retrained with more available data in the future. Moreover, to minimize predictive uncertainty, we deliberately avoided data imputation, ensuring that no prediction is rendered unless all required feature values are present to inform the physician’s judgment. The system was designed to issue error or warning messages when it encounters missing data or unstructured entries that do not meet the input specifications. However, we recognize that this “complete-case” requirement may limit the real-world utility of our models in clinical settings with less structured data entry. To address this, future iterations will explore multivariate imputation by chained equations, enabling the model to have robust performance in the presence of missing data and mitigating current utility restrictions while maintaining diagnostic reliability.

Seventh, the inferred time savings associated with the model are based on a retrospective comparison of median LoS. These findings are subject to confounding by indication, as the single-test cohort likely represented patients with lower perceived clinical risk. While the results suggest a trend toward improved efficiency through reduced repeat testing, they should be interpreted as the theoretical potential of a single-sample strategy. The actual impact on LoS in prospective deployment may vary depending on clinical judgment, ED crowding, and institutional protocols [40], and cannot be definitively predicted from this observational data.

Eighth, the assessment of potential improvements in ED workflow following model implementation was based on retrospective data. Future prospective studies, ideally incorporating collaboration with experts in operations research, are needed to evaluate the real-world impact of integrating the model into clinical practice.

Finally, the testing AUPRC for the high-risk (GA) group was only 0.391. Given the GA prevalence of only 4.6% in our test cohort, this performance represents a significant improvement over random classification; however, it also reflects a lower PR balance for this rare but critical class. While our model prioritizes a high NPV (>99.5%) to ensure patient safety, this emphasis may result in a higher rate of false positives. This conservative bias could potentially lead to increased resource use and diagnostic observation in the ED. Future work will focus on incorporating dynamic clinical features to enhance precision without compromising the model’s high safety threshold.

The AI models developed in this study serve as a preliminary proof-of-concept for providing CDS to physicians managing ACP. The multiclassification model we developed achieved 0.941 sensitivity for identifying patients with ACS and 0.837 sensitivity and 0.911 PPV for identifying nonsevere patients. Despite many limitations, these results suggest that integrating AI-based multiclassification algorithms into ED workflows has the potential to enhance risk stratification and optimize resource use. However, we recognize that translating these findings into real-world practice involves significant sociotechnical, medical-legal, and workflow challenges. Ultimately, the objective of this work is to develop tools for supporting ED physicians to accurately evaluate ACP within a broad range of clinical guidelines. To enhance predictive performance and broaden model utility, future iterations will focus on four key areas: (1) retraining models using symptom-onset timing and high-impact features, which may include descriptors extracted from digitized ECG signals and nursing notes; (2) integrating data imputation strategies, such as multivariate imputation by chained equations, to handle missing data; (3) developing dedicated models tailored to specific biomarker profiles, such as patients with low eGFR due to renal failure; and (4) validating the models using larger, diverse, multicenter datasets.