Ethical approval and consent for participation are not applicable, as the research was conducted using publicly available, anonymized data from the NHAMCS-ED (National Hospital Ambulatory Medical Care Survey–Emergency Department) dataset.

We used data from the 2021 NHAMCS-ED dataset [14], a nationally representative dataset that captures a wide range of information about visits to EDs across the United States. NHAMCS-ED is administered by the National Center for Health Statistics and uses a multistage probability sampling design to ensure generalizability to the national population [15]. The dataset includes visit-level records detailing patient demographics, clinical characteristics, reasons for visits, procedures, diagnostic tests, medications, and disposition. For this study, the sample was restricted to adult patients aged 18 years or older. The study excluded visits with missing data on our primary outcome of interest—laboratory test use—as well as records with critical missing values for key predictor variables. The final analytic cohort comprised all eligible adult ED visits for which complete information on laboratory testing and associated predictors was available.

Laboratory test use was operationalized as a binary outcome, indicating whether one or more laboratory tests from a predefined set of commonly ordered assays were prescribed during an ED visit. Using available NHAMCS-ED variables and informed by prior literature, we identified 20 laboratory tests and panels spanning multiple organ systems and clinical indications, such as metabolic, hematologic, renal, hepatic, and coagulation-related tests.

For each visit, test ordering was coded as “1” if at least one of these tests or panels was ordered and “0” otherwise. Visits involving multiple tests or panels were treated as positive events without additional weighting, as the objective was to capture the occurrence of laboratory evaluation rather than test volume, sequencing, or clinical appropriateness. Because there is no widely accepted approach for encoding the complex combinations and ordering patterns of laboratory tests in large-scale ED datasets, this binary operationalization is a pragmatic, scalable strategy that avoids arbitrary weighting while supporting robust predictive modeling.

Structured predictors were drawn from multiple domains within the NHAMCS dataset. These included demographic variables (eg, age, sex, race, or ethnicity), visit characteristics (eg, day of the week, time of arrival, arrival by ambulance, and mode of arrival), and vital signs (eg, heart rate, blood pressure, respiratory rate, temperature, oxygen saturation, and pain level).

We also included a comprehensive set of medical history variables, coded as binary indicators for the presence of specific chronic or acute conditions. These included Alzheimer disease or dementia, asthma, cancer, cerebrovascular disease, chronic kidney disease, chronic obstructive pulmonary disease, congestive heart failure, coronary artery disease, diabetes mellitus (types I and II), end-stage renal disease, HIV/AIDS, hyperlipidemia, hypertension, obesity, obstructive sleep apnea, osteoporosis, depression, substance abuse or dependence, and thromboembolic conditions such as deep vein thrombosis or pulmonary embolism.

Additional structured features included insurance type (private, Medicare, Medicaid, Children's Health Insurance Program, uninsured, or other); residence type (private home, nursing home, homeless, or other); whether the visit was a follow-up or related to an earlier episode of care; and whether the visit involved injury or trauma, poisoning or overdose, or adverse effects of treatment. These structured features were selected based on prior literature, clinical rationale, and exploratory data analysis.

Prior to modeling, continuous variables were standardized using the StandardScaler method to facilitate model convergence and ensure comparable feature-importance estimates across models.

Missing data analysis revealed that, except for the pain score variable (with 38.3% missing), all structured features exhibited low to moderately low levels of missingness, that is, less than 10% (Multimedia Appendix 1). To assess the plausibility of the missing-at-random assumption, we examined patterns of missingness in relation to laboratory test use and observable visit characteristics for the pain score variable. Notably, the missingness of the pain score variable was not associated with laboratory testing rates (ie, 59.6% for observed vs 59% for missing), and was more frequent among ambulance arrivals, consistent with higher-acuity encounters in which clinical priorities may precede formal pain documentation.

For other administrative and visit-level variables, differences in laboratory testing rates between missing and observed groups were very low to modest. Larger differences were observed for vital signs, such as temperature, oxygen saturation, and pulse, reflecting ED workflows in which these parameters are preferentially measured in higher-acuity encounters, which are also more likely to undergo laboratory evaluation. Taken together, these patterns suggest that missingness in structured variables is largely driven by observable clinical context and documentation practices rather than outcome-dependent mechanisms, thereby supporting the use of median imputation under the missing-at-random assumption.

The NHAMCS-ED dataset includes two primary unstructured text fields: (1) the chief complaint, which captures the patient’s primary reason for visiting the ED, and (2) the reason-for-visit injury text, which describes any injury-related circumstances. These free-text fields provide rich clinical context often unavailable in structured variables and are frequently used by ED staff to understand the patient’s presenting condition.

To process these fields, we applied a text-cleaning pipeline to enhance consistency and remove irrelevant noise [16]. This included converting all text to lowercase, removing punctuation and numerals, stripping whitespace, and filtering out common stopwords (eg, “and,” “the,” “of”) using a predefined list from the Natural Language Toolkit [17]. The resulting cleaned text provided the input for our NLP pipeline.

We used a pretrained Bidirectional Encoder Representations from Transformers (BERT) model from the HuggingFace Transformers library to encode the cleaned text [18]. BERT is a state-of-the-art transformer-based language model that learns deep contextual relationships between words through bidirectional attention. The BERT-base-uncased model and its tokenizer were used to convert each text entry into input tokens compatible with the BERT model architecture [19].

These tokenized inputs were then passed through BERT to extract sentence-level embeddings—dense vector representations capturing the semantic content of the chief complaint and injury text. Performance was robust to alternative text-embedding pooling ([CLS], also known as classification, token vs mean pooling). These embeddings, typically 768-dimensional vectors, served as the unstructured input features for downstream modeling. They enabled us to leverage the rich semantic meaning encoded in patient narratives, augmenting the structured clinical features.

To assess whether our reliance on the [CLS] embedding affected model performance, we also evaluated an alternative, widely used aggregation strategy: mean pooling over token embeddings [20]. The 2 approaches yielded highly comparable results in the gradient boosting classifier (receiver operating characteristic curve, AUC=0.83 for [CLS] vs 0.84 for mean pooling), indicating that sentence-level performance was robust to the choice of pooling method. Given the comparison result, we retained the [CLS] representation as our primary specification, consistent with standard BERT practice for sequence-level encoding [21].

To identify factors independently associated with laboratory test use, a logistic regression with Lasso regularization was performed for variable selection [22]. This approach penalizes less informative predictors and retains only those with the strongest associations, helping reduce overfitting and improving model interpretability. After selecting variables with nonzero coefficients from the Lasso model, a standard multivariable logistic regression was fitted using the selected predictors to estimate adjusted odds ratios (ORs) and 95% CIs. A forest plot was generated to visualize the direction and strength of associations. Variables with ORs above 1 were associated with higher odds of laboratory testing, while ORs below 1 indicated a reduced likelihood. This analysis highlighted key clinical and demographic drivers of test-ordering decisions in the ED.

To predict laboratory test use during ED visits, we developed 4 supervised ML models: logistic regression, random forest, gradient boosting, and extreme gradient boosting. These models were selected to capture both linear and complex nonlinear relationships among predictors. Logistic regression served as a benchmark model due to its interpretability, while the ensemble-based models (random forest, gradient boosting, and extreme gradient boosting) offered greater flexibility in capturing variable interactions and improving classification performance.

Each model was applied to four configurations of input data: (1) structured data only, including demographics, vitals, medical history, and visit details; (2) unstructured text data only, using BERT-generated embeddings from chief complaints and reasons for visit; (3) combined structured and unstructured data; and (4) a mean probability model, which averaged predicted probabilities from the structured-only and unstructured-only models.

Structured data were preprocessed using median imputation and standardized for modeling. Unstructured data were cleaned, tokenized, and processed through a pretrained BERT model to generate contextual embeddings. All models were trained and evaluated using 5-fold cross-validation to ensure robust performance estimates across data splits.

Model performance was evaluated using standard classification metrics: area under the receiver operating characteristic (ROC) curve (AUC) [23], accuracy, precision, recall (sensitivity), specificity, and F₁-score [24]. AUC was the primary measure of discrimination, reflecting the model’s ability to distinguish between visits with and without laboratory testing. Precision and recall offered insights into the model’s accuracy in predicting test use, with the F₁-score providing a balanced measure of both.

We also examined specificity to assess how well the models avoided false positives—cases where testing was not conducted but was predicted as such. Optimal classification thresholds were determined using the point on the ROC curve closest to the top-left corner.

To aid interpretation, we visualized logistic regression coefficients using forest plots and generated word clouds from unstructured data to highlight common terms associated with laboratory testing decisions.

As an important extension of our analysis, feature contributions were quantified using Shapley additive explanations (SHAP) [25]. For tree-based models, TreeExplainer was used and for logistic regression [26], the linear explainer was used with the model’s logit link [27]. To synthesize the unstructured input (ie, BERT embeddings), we computed SHAP values and aggregated them into a single textual feature by summing the mean absolute SHAP values across all dimensions, reported as Combined_Chief_Complaint.

Among adult ED visits in the 2021 NHAMCS-ED dataset, 7784 (59.3%) visits included laboratory test orders. Significant differences were observed across nearly all demographic and clinical variables between patients who received laboratory tests and those who did not (Table 1). Patients who underwent laboratory testing were more likely to be female (4341/7784, 55.8% vs 2741/5331, 51.4%) and older, with 2298 out of 7784 (29.5%) aged 65 years or older compared to 834 out of 5331 (15.6%) in the no-test group. The proportion of White patients was slightly higher in the tested group (4779/7784, 61.4% vs 2958/5331, 55.5%), while Black and Hispanic patients were slightly underrepresented. Laboratory-tested patients were more likely to reside in nursing homes (208/7784, 2.7% vs 58/5331, 1.1%) and to be insured through Medicare (2296/7784, 29.5% vs 905/5331, 16.9%), while the no-test group had higher proportions of Medicaid (1766/5331, 33.1% vs 2033/7784, 26.1%) and uninsured individuals (480/5331, 9% vs 461/7784, 5.9%). Patients with laboratory testing were more frequently transported by ambulance (2012/7784, 25.8% vs 682/5331, 12.7%) and less likely to have injury-related visits (1015/7784, 13% vs 2137/5331, 40%). Clinical severity indicators also differed: higher proportions of laboratory-tested patients had abnormal vital signs, including heart rates of more than 90 bpm (3110/7784, 39.9% vs 1712/5331, 32.1%), diastolic blood pressure of less than 60 mm Hg (536/7784, 6.9% vs 218/5331, 4.1%), oxygen saturation of less than 95% (996/7784, 12.8% vs 297/5331, 5.6%), and respiratory rates of more than 20 breaths/minute (914/7784, 11.7% vs 210/5331, 3.9%). Patients who received laboratory tests had a higher prevalence of chronic conditions, including hypertension (3116/7784, 40% vs 1183/5331, 22.2%), diabetes (type II: 936/7784, 12% vs 301/5331, 5.6%), coronary artery disease (839/7784, 10.8% vs 192/5331, 3.6%), chronic kidney disease (537/7784, 6.9% vs 100/5331, 1.9%), congestive heart failure (569/7784, 7.3% vs 110/5331, 2.1%), and chronic obstructive pulmonary disease (684/7784, 8.8% vs 231/5331, 4.3%). Other conditions significantly more common in the laboratory-tested group included cancer, dementia, stroke, end-stage renal disease, and hyperlipidemia.

Figure 1 displays adjusted odds ratios (ORs) and 95% CIs for predictors of laboratory test use during ED visits, derived from Lasso-selected logistic regression models. Older age, particularly aged 65 years and older (OR 2.30, 95% CI 2.01‐2.62), and arrival by ambulance (OR 2.31, 95% CI 2.07‐2.58) were among the strongest predictors of laboratory test use. Patients with chronic conditions such as hypertension (OR 1.87, 95% CI 1.68‐2.08), type II diabetes (OR 1.69, 95% CI 1.47‐1.93), chronic kidney disease (OR 2.12, 95% CI 1.75‐2.56), and cancer (OR 1.85, 95% CI 1.48‐2.30) had significantly higher odds of receiving laboratory tests. In contrast, visits related to injury or trauma were associated with substantially lower odds of testing (OR 0.39, 95% CI 0.35‐0.43). Elevated heart rate (>90 bpm) (OR 1.32, 95% CI 1.20‐1.45), low diastolic blood pressure (<60 mm Hg) (OR 1.36, 95% CI 1.15‐1.60), respiratory rate of more than 20 breaths/minute (OR 1.66, 95% CI 1.37‐2.02), temperature of more than 38°C (OR 1.87, 95% CI 1.32‐2.65), and oxygen saturation of less than 95% (OR 2.38, 95% CI 2.03‐2.78) were all significantly associated with increased odds of laboratory testing, reflecting the influence of clinical instability on diagnostic intensity.

Figure 2 illustrates the ROC curves for the 4 ML models developed to predict laboratory test use during ED visits. These models included (1) a structured-only model, which used demographic, clinical, and visit-level features; (2) an unstructured-only model, which used BERT-derived embeddings from free-text chief complaints and reason-for-visit narratives; (3) a combined model, which integrated both structured and unstructured inputs; and (4) a mean probability model, which averaged the predicted probabilities from the structured-only and unstructured-only models without retraining.

Among the models, the combined model demonstrated the highest predictive performance, with an AUC of 0.83, indicating excellent discrimination between visits with and without laboratory test orders. This performance reflects the model’s ability to capture both quantifiable patient characteristics and nuanced clinical context embedded in free-text narratives. The structured-only model achieved a slightly lower AUC of 0.78, suggesting that structured data alone provided a strong baseline for prediction but lacked the richness conveyed in narrative data. The unstructured-only model, based entirely on transformer-derived text embeddings, yielded an AUC of 0.74. While lower than the structured and combined models, it still demonstrated that free-text chief complaints and injury descriptions carry meaningful predictive signals for test use, especially in the absence of structured data. The mean probability model—a late-fusion ensemble averaging outputs from the structured-only and unstructured-only models—achieved an AUC of 0.81, performing better than either source alone but slightly below the fully integrated combined model. This result underscores the benefit of model ensembling, even without retraining or direct feature integration. All detailed performance metrics are presented in Table 2.

SHAP analysis findings were consistent with logistic regression. The Combined_Chief_Complaint was the most influential feature across all models. Similar findings were observed across models; the most influential features included age, a visit related to injury, poisoning, or an adverse effect of medical or surgical treatment, ambulance arrival, and vital signs (hypertension, chronic kidney disease, and diabetes). Table 3 presents all SHAP values across models.

This study developed and evaluated ML models to predict whether general laboratory tests are ordered during ED visits and found that combining structured EHR variables with unstructured clinical text yields the strongest predictive accuracy. Across model families, the combined approach consistently outperformed models built on structured-only or unstructured-only inputs, underscoring the complementary value of integrating quantifiable patient factors with the contextual nuance present in free-text narratives. Notably, such a finding builds on and further affirms the existing methodological literature suggesting the potential of combining structured and unstructured data [28,29].

Methodologically, our work extends prior studies that either relied exclusively on structured EHR data or incorporated unstructured text using basic NLP techniques. By leveraging contextualized language model embeddings (BERT) alongside structured features and comparing structured-only, unstructured-only, and combined inputs within a common framework, we demonstrate the added value of free-text clinical narratives for laboratory-order prediction at scale. In 5-fold cross-validation, the combined model achieved an AUC of 0.83, compared with 0.78 for structured-only, 0.74 for unstructured-only, and 0.81 for a simple late-fusion mean-probability ensemble, indicating that feature-level integration captures informative cross-modal interactions beyond the ensemble alone. Performance was robust to alternative text-embedding pooling strategies ([CLS] vs mean pooling), suggesting that gains are not an artifact of a particular representation choice. Model interpretability analyses were concordant across methods: the aggregated text-embedding feature and clinically intuitive predictors (eg, age, ambulance arrival, injury indicator, comorbidities, and vital signs) were among the most influential contributors, consistent with associations observed in Lasso-selected logistic regression.

Across all ML model families, SHAP analyses demonstrated a highly consistent set of influential predictors, including patient age, arrival by ambulance, abnormal vital signs (notably tachycardia, tachypnea, hypoxia, and fever), chronic comorbidity burden, and injury-related visit status. These findings align closely with established emergency medicine literature showing that older age, higher acuity of arrival, and physiologic instability are central drivers of diagnostic intensity and laboratory use in ED populations [30,31]. Large observational studies and multicenter cohort analyses have consistently demonstrated that derangements in initial vital signs are strongly associated with downstream diagnostic evaluation, admission, and short-term adverse outcomes, particularly among older and medically complex patients [32]. Similarly, chronic conditions such as diabetes, hypertension, and chronic kidney disease have been shown to increase clinicians’ diagnostic vigilance due to higher baseline risk, atypical presentations, and concern for occult metabolic or infectious pathology, reinforcing their consistent importance across predictive models [33].

In contrast, injury-related or trauma-related visits were consistently associated with a lower predicted likelihood of laboratory testing, reflecting well-described trauma workflows that emphasize imaging, procedural assessment, and protocolized resuscitation over broad laboratory evaluation in the absence of physiologic instability [34,35]. Prior trauma-focused studies have shown that routine laboratory testing in low-to-moderate-severity trauma often yields limited actionable information, supporting more selective testing strategies [36]. Notably, the aggregated unstructured text feature (Combined Chief Complaint) emerged as the most influential predictor across all models, underscoring the critical role of free-text documentation in capturing early diagnostic reasoning. Recent explainable machine-learning studies in emergency care have demonstrated that clinical narratives encode nuanced contextual signals—such as concern for infection, metabolic disturbance, or cardiorespiratory compromise—that may precede or extend beyond structured data in shaping test-ordering behavior [28,33]. Together, these findings indicate that integrating unstructured clinical text not only improves predictive performance but also closely mirrors real-world clinical reasoning during early ED decision-making.

These findings have practical implications for diagnostic stewardship in ED settings. Because unnecessary or default laboratory testing contributes to avoidable costs, patient burden, and workflow inefficiency, predictive models that identify encounters with a lower likelihood of testing could serve as a lightweight prompt for clinicians to reassess routine or defensive ordering. In this way, model outputs can support more efficient, evidence-informed, and patient-centered care while preserving clinician judgment [37].

The results also highlight the use of transformer-based language models in clinical prediction tasks: contextual embeddings capture linguistic complexity and semantic relationships in chief complaints and triage narratives, such as early concern for infection, hemodynamic instability, or metabolic disturbance, which may precede and inform ordering decisions in ways not fully represented by structured data [38]. As large language models trained on clinical corpora continue to evolve, there is potential to further improve performance, interpretability, and operational usability for diagnostic stewardship applications.

Beyond model performance, we observed differences in laboratory testing by race and insurance status that mirror longstanding variations in diagnostic intensity across US EDs. Prior work has documented that minoritized racial groups often receive fewer diagnostic tests and procedures, even after accounting for presenting features, while Medicare-insured patients–typically older with a higher comorbidity burden—more often undergo extensive evaluation, and uninsured or Medicaid-insured patients experience reduced diagnostic intensity. Our findings align with this broader literature and underscore the importance of examining how predictive systems may interact with existing disparities in care [30,39]. Future development should incorporate subgroup performance assessment and formal fairness analyses to mitigate the risk of reinforcing inequities [40,41].

Generalizability also warrants consideration. Although NHAMCS-ED provides a nationally representative picture of US emergency care, practice patterns, documentation structures, and diagnostic workflows can differ across individual hospitals, integrated health systems, and international ED settings. Variations in staffing models, triage protocols, and resource availability may influence both baseline testing rates and the predictors of ordering decisions. Accordingly, site-specific validation and prospective assessment are needed prior to operational deployment to ensure reliable performance across diverse clinical environments.

Finally, the clinical relevance of use prediction depends on its linkage to diagnostic quality and outcomes. While our models accurately predict the occurrence of laboratory testing, they do not evaluate appropriateness or downstream benefits (eg, diagnostic yield, treatment changes, ED length of stay, admission, and 72-h revisit). As such, this work should be viewed as a use-prediction study rather than an assessment of diagnostic quality. Prospective studies that pair deployment with adjudicated appropriateness criteria (eg, guideline-concordant ordering and choosing wisely targets) and patient-relevant outcomes are needed to establish clinical utility beyond use reduction.

Translating these models into decision support also requires attention to implementation details. Interpretability is essential for clinician trust; concise, case-level explanations (eg, SHAP-based summaries or simplified feature highlights) can clarify why a prediction was generated. User-interface design should minimize workflow disruption by surfacing predictions at natural decision points (eg, triage or during order entry) and offering actionable, context-aware cues rather than prescriptive directives. To avoid exacerbating alert fatigue, signals should be delivered judiciously, preferably through passive or noninterruptive displays with configurable thresholds and governance oversight. Addressing these considerations will be critical to realizing the potential of combined structured–unstructured models to support diagnostic stewardship in emergency care.

Despite the promising results, several limitations must be acknowledged. First, though nationally representative, the NHAMCS-ED dataset is cross-sectional, thus limiting our ability to further evaluate the long-term pattern of laboratory testing in future ED admissions, if any. Second, while the models performed well in predicting laboratory orders, they do not assess clinical appropriateness. Future work should aim to distinguish between necessary and unnecessary testing to directly address overuse. Third, laboratory test use was operationalized as a binary outcome, which does not capture the complexity of test selection, test combinations, or sequencing during ED encounters. While this approach supports scalable modeling and avoids arbitrary weighting of heterogeneous tests, more granular representations of laboratory ordering behavior may yield additional insights in future studies. Fourth, integrating predictive tools into clinical workflows requires tailored and setting-specific design, critical provider engagement, and diligent ethical oversight to ensure that the integration of ML models supports rather than replaces clinician judgment. Fifth, this study did not formally assess algorithmic fairness or evaluate subgroup performance using established fairness metrics (eg, disparate impact, equal opportunity, or calibration across race or ethnicity or insurance groups). Given the observed variation in laboratory testing across demographic and insurance strata, future research should incorporate formal fairness analyses to ensure that predictive models do not inadvertently reproduce or amplify existing inequities in diagnostic care. Sixth, this study did not assess the clinical appropriateness or downstream benefit of laboratory testing; as such, results reflect the prediction of test occurrence rather than diagnostic quality or impact on outcomes. Finally, the study could enhance its applicability by outlining practical steps for real-world implementation and validation within hospital environments. Specifically, it should address strategies for effectively integrating predictive insights into existing clinical workflows in a manner that is nonintrusive yet informative for clinicians, ensuring these models enhance rather than hinder decision-making processes.

In summary, this study confirms the feasibility and added value of using combined structured and unstructured EHR data to predict laboratory test orders in ED settings. Such models can inform more intelligent diagnostic workflows by capturing quantifiable risk factors and nuanced clinical reasoning. Future work should focus on the real-time implementation and validation of diagnostic stewardship, aiming to reduce health care waste and improve patient outcomes in acute care environments.