Many prediction tools have been created to assess the risk of undiagnosed diabetes and related outcomes such as impaired glucose tolerance, prediabetes, risk of future diabetes, and hyperinsulinemia. Most of these tools are not practical in the setting of the electronic health record (EHR) because they include predictor variables not readily available in structured formats [1-20]. Examples of impractical variables include waist circumference, fasting time, physical activity, review of systems, diet, pregnancy-related variables, and detailed ethnicity. Tools typically leverage fasting glucose level as a predictor, which is simple to obtain in practice, but documentation of fasting time in a structured fashion in EHRs is generally absent. The authors identified two tools that accurately predict the presence of diabetes using structured variables routinely present in the EHR [21,22].

Current guidelines from the United States Preventive Services Task Force (USPSTF) recommend screening for abnormal blood glucose in adults aged 40 to 70 years who are overweight or obese. The USPSTF acknowledges that patients with other high-risk characteristics (eg, family history of diabetes, personal history of gestational diabetes) may need to be screened sooner but this is left up to the physician’s discretion [23]. The guidelines published by the American Diabetes Association (ADA) recommend glucose screening of adult patients with an elevated body mass index (BMI; ≥25 kg/m²) plus another risk factor (eg, hypertension, physical inactivity, family history of diabetes) at any age and for all patients beginning at 45 years of age at 3-year intervals [24].

Current approaches do not take advantage of advanced statistical modeling. Hyperglycemia risk prediction with simultaneous consideration of numerous independent variables and nonlinear effects is statistically ideal in the context of a multifactorial pathology. Creating strict cutoffs of individual variables or combinations of a limited number of variables for clinical guidelines does not take advantage of what can now be reasonably achieved. The USPSTF and ADA guidelines encourage physician judgment in the application of glucose screening but do not provide specific guidance. Simplified classification methods used in cancer have been notoriously poor at discriminating between high- and low-risk patients [25]. Moreover, many existing models for predicting hyperglycemia risk are also likely reducing their prediction accuracy by categorizing continuous variables, which reduces granularity and may miss potentially complex associations between a continuous variable and the outcome. This issue was highlighted by Kattan [26] when he showed that traditional regression techniques that incorporated restricted cubic splines to reduce linearity assumptions were found to produce more accurate risk prediction models when compared with classification methods such as classification and regression trees and artificial neural networks.

Predicting the date of onset of hyperglycemia is difficult due to the lack of symptoms early in the course of the disease and inconsistent testing and/or documentation in clinical practice (particularly in a structured fashion). Previous studies indicate that the onset of type 2 diabetes frequently occurs more than 5 years before diagnosis [27,28]. In contrast, blood measurements of glycated hemoglobin (HbA_1c) provide an easy and accurate method for determining current mean glycemia over the previous 8 to 12 weeks without the need for fasting. HbA_1c testing is standardized according to specifications defined by the National Glycohemoglobin Standardization Program (NGSP). HbA_1c levels are the primary blood marker used for guiding the management of type 2 diabetes, and the ADA has approved HbA_1c testing for diabetes screening [24]. The increasing use of HbA_1c as a screening tool in patients without prediabetes or diabetes provides data for prediction modeling from EHR records.

The authors strongly believe that the identification of patients with elevated HbA_1c is important clinically despite previous studies that have not shown a mortality benefit from screening for diabetes [29]. The early detection of elevated blood sugar can have other significant benefits:

Therefore, an accurate tool for predicting the current probability that a specific patient has an elevation in HbA_1c levels would constitute a major advancement in finding patients with the most probable need of screening interventions. To address this gap, we created a calculator for predicting the probability that a given patient with no history of diabetes or elevated blood sugar currently has an elevated HbA_1c value (≥5.7%). This cutoff was chosen because it corresponds with the current guidelines published by the ADA that indicate values <5.7% are considered to be normal. Importantly, the calculator presented in this paper was restricted only to structured variables typically available in EHRs. This focus on common structured variables will enable the tool to be integrated into EHRs for implementation.

This study was conducted on all adult patients who have undergone HbA_1c testing prior to evidence of hyperglycemia (random blood sugar ≥200 mg/dL), any diabetes-related diagnostic code, or prescription for an antihyperglycemic medication. Data were extracted from the Epicare EHR at Wake Forest Baptist Medical Center in Winston-Salem, North Carolina, for the dates between September 2012 and September 2016. The study was approved by the institutional review board and granted a waiver of informed consent. Data were limited to structured data located in these areas of the EHR: encounter diagnoses, problem list, past medical history, procedures, prescriptions, vital signs, demographics, social history, and laboratory values. Candidate predictor variables were chosen based on their theoretical association with hyperglycemia. Textbox 1 shows a list of the candidate predictor variables included in the complete statistical model.

Independent variables were defined on the date of the HbA_1c of interest. For missing continuous variables (eg, systolic blood pressure), the most recent prior value was used instead. Patients completely lacking values for independent variables were excluded. The investigators did not impute missing data because it was felt that imputation would not be appropriate at the point of implementation. Comorbidities were considered to be present if the patient had any structured instances of the diagnostic code on or before the date of the first HbA_1c. Medications with start dates on or before the date of the HbA_1c and end dates on or after the date of HbA_1c were considered to be active. Medication order dates were used when the start dates were missing. Medications missing both start and order dates were excluded. Medication categories (eg, antihyperglycemics) were provided by First Databank Inc. Multiple logistic regression was used to model the association between the independent variables and the outcome of HbA_1c ≥5.7%. Continuous variables were fit using restricted cubic splines with 3-knots. Due to collinearity, the model could not be fit with the simultaneous inclusion of serum non–high-density lipoprotein and high-density lipoprotein. Therefore, high-density lipoprotein was removed from the complete model. The model was reduced using Harrell’s model approximation method [39]. For parsimony, the diagnosis of obesity variable was removed after variable selection. The diagnosis had little impact on the prediction accuracy and was redundant since BMI is also in the model. The reduced model was compared head-to-head with the calculators created by Baan and Griffin [21,22]. The head-to-head comparisons were performed using 10-fold cross-validation in order to calculate the bias-adjusted prediction accuracy of the new model. Prediction model metrics included measures of discrimination (concordance statistic), calibration (calibration curves), and decision curves [40]. Statistical analyses were performed in R version 3.2.5 (R Foundation for Statistical Computing) using the rms (Regression Modeling Strategies) package.

The record search identified 22,635 patients for model building and validation of which 26% were found to have an elevated HbA_1c (≥5.7%). Figure 1 shows the number of patients included and excluded from the model building.

The final model included the following 8 variables ordered from most important to least important: age, BMI, random glucose, race, serum non–high-density lipoprotein, serum total cholesterol, estimated glomerular filtration rate (eGFR), and smoking status. Table 1 shows descriptive statistics for the variables included in the final model by HbA_1c results. As expected, patients found to have elevated HbA_1c levels were older, had higher BMI, lower eGFR, and higher random glucose values.

The coefficients along with instructions for calculating the probability of an elevated HbA_1c (≥5.7%) and sample calculations for 2 patient scenarios are shown in Multimedia Appendix 2.

The 3 models were compared in their ability to accurately rank patients according to risk as measured by the concordance statistic (c-stat) and bias-adjusted using 10-fold cross-validation. The current model (c-stat 0.765, 95% CI 0.762 to 0.769) demonstrated statistically significant improvements in discrimination when compared to the models created by Baan (c-stat 0.637, 95% CI 0.633 to 0.641) and Griffin (c-stat 0.668, 95% CI 0.665 to 0.672).

The calibration curve shown in Figure 2 reveals that the current model is well calibrated. The predicted probabilities tend to overestimate risk at the right tail of the distribution, but the wide confidence intervals allude to the scarcity of the data at these extreme high levels of risk. Error bars represent the 95% confidence interval around the point estimate.

Decision curves are displayed in Figure 3 and also demonstrate the superiority of this model. Our model shows a net benefit up to a probability of 0.73 for an elevated HbA_1c (≥5.7%) without significant net harms above this threshold. This model confers a net benefit that is equal to or greater than the net benefit offered by the other models at all probability thresholds.