Prediction of Insulin Resistance in Nondiabetic Population Using LightGBM and Cohort Validation of Its Clinical Value: Cross-Sectional and Retrospective Cohort Study

Introduction

Insulin resistance (IR) refers to the reduced sensitivity or responsiveness of target organs such as the liver, skeletal muscles, or adipose tissue to insulin. In order to maintain normal blood glucose levels, the body compensates by secreting increased amounts of insulin [1]. As a condition that can exist before the onset of type 2 diabetes, IR is not only one of the key mechanisms underlying the development of diabetes but also a major risk factor for various diseases [2]. According to the International Diabetes Federation, it is estimated that by 2045, there will be 783.2 million people affected globally, with the vast majority having type 2 diabetes [3]. Furthermore, IR is even more common. Data from the National Health and Nutrition Examination Survey in the United States show that approximately 40% of adults aged 18‐44 years have IR [4]. A study in China revealed that the standardized prevalence of IR in adults aged 25 years or older is 29.22% [5]. By scientifically assessing and managing IR, it is theoretically possible to effectively control the onset of diabetes. The first step is to identify IR. The high insulin clamp with normal blood glucose levels is considered the gold standard for assessing insulin sensitivity [6]. However, in practice, this method is time-consuming, expensive, and difficult to apply widely. The Homeostasis Model Assessment of Insulin Resistance (HOMA-IR; fasting glucose [mmol/L]×fasting insulin [μU/ml]/22.5) is regarded as an acceptable method for evaluating IR [7]. However, fasting insulin, one of its components, is not a routine test, making it difficult to obtain in community and grassroots settings. Based on laboratory indicators and lifestyle characteristics that are correlated with the occurrence of IR and are easily accessible, it is crucial to explore more convenient ways to predict the previous probability of IR in individuals. This approach could guide more precise IR screening and has the potential for prevention and control within the large population of China.

Machine learning (ML), with its data-driven methods, has already shown good predictive value in health issues such as arteriosclerosis [8], fatty liver [9], diabetes [10], and hypertension [11] . Obesity, dyslipidemia, lack of exercise, and sedentary lifestyles are considered risk factors for metabolic homeostasis [12]. By applying daily habits, basic information, and simple blood tests as features, we can build machine learning models in health check-up populations that are representative of the general population. The model of the ensemble learning algorithms based on decision trees, such as random forest (RF), Light Gradient Boosting Machine (LightGBM), Extreme Gradient Boosting (XGBoost), Gradient Boosting Machine (GBDT), and CatBoost, using multiple decision trees for predictions. They are capable of capturing complex nonlinear relationships, handling high-dimensional data, performing automatic feature selection, and mitigating the risk of overfitting through various mechanisms. In addition, these models offer strong scalability and are suitable for large-scale datasets, with most models having built-in overfitting prevention capabilities. These models can effectively predict IR and identify key predictive factors that are crucial for individual intervention. In addition, a retrospective cohort study can be used to determine the accuracy and practical value of these model results in distinguishing the onset of diabetes.

Methods

Study Population and Data Sources

The data used in this study were sourced from the electronic medical records of the Health Management Center at Xiangya Third Hospital, Central South University. The dataset includes participants who underwent physical examinations and questionnaire surveys at the Health Management Center between January 2018 and August 2022. A total of 17,287 adults with normal fasting blood glucose (FBG) were included, while individuals with FBG ≥6.1mmol/L or those with a history of abnormal glucose tolerance or diabetes were excluded.

The subsequent retrospective cohort consisted of individuals who completed their first health check-up at the Health Management Center of Xiangya Third Hospital between January 2017 and January 2019. These participants were followed for 5 years, with those diagnosed with diabetes at external hospitals or those who had completed ≥2 times FBG tests at the hospital during the follow-up period being included. Participants were required to be aged 18 years or older and have FBG <6.1mmol/L at the time of their first check-up. They also had to complete baseline assessments and surveys. Exclusion criteria included a history of abnormal glucose tolerance, diabetes, or gestational diabetes, as well as a history of tumors, pregnancy, or autoimmune diseases; and those who had used glucocorticoids or other relevant medications within the past month.

Using an IR prediction model, individuals in the cohort were classified as low-risk or high-risk based on their baseline data. Changes in FBG were then monitored. A FBG ≥7 mmol/L within 5 years or a diagnosis of diabetes at an external hospital was considered a positive result. A negative result was defined as maintaining normal FBG levels without a diabetes diagnosis at an external hospital over the 5-year follow-up period. Furthermore, by matching the ID card numbers, we excluded the participants who repeatedly appeared in the training set and the test set. Ensure this cohort was the different from the dataset used for model development.

Characteristics and Definitions

The dataset includes a total of 102 features, which encompass physical examination measurements such as age, height, weight, BMI, waist circumference (WC), systolic blood pressure (SBP), and diastolic blood pressure (DBP). Blood test indicators include FBG, liver function markers (alanine aminotransferase [ALT], aspartate aminotransferase [AST], total bilirubin [TBIL], direct bilirubin [DBIL], serum albumin [ALB], serum globulin [GLO], albumin/globulin ratio [A/G], total proteintotal [TP], bile acids [TBA]), kidney function markers (serum creatinine [Sr], blood urea nitrogen [BUN], uric acid [UA], and lipid profile markers (total cholesterol [TC], triglycerides (TG], high-density lipoprotein cholesterol [HDL-C], low-density lipoprotein cholesterol [LDL-C]. In addition, information from questionnaires was collected, including past disease diagnoses, lifestyle factors (dietary preferences, smoking status, alcohol consumption, exercise habits, and work status), psychological status, sleep patterns, and health literacy related to basic medical knowledge. Insulin sensitivity was assessed using HOMA-IR=fasting glucose (mmol/L)×fasting insulin (μU/ml) / 22.5. A cut-off value of 2.69 was used, with a value ≥2.69 considered indicative of IR status [5].

Data Processing and Statistical Analysis

We excluded features with more than 30% missing data. For features with less than 30% missing data, continuous variables were imputed using the mean, while categorical variables were imputed using the mode. Ultimately, 90 features and 16,411 samples were retained. The entire dataset was randomly split into a training set (13,128/16411, 80% samples) and a test set (3283/16411, 20% samples), with 2782 samples categorized as IR.

For statistical analysis, continuous data that conform to a normal distribution were reported as means (SDs), otherwise, were reported as quartiles. Chi-square tests were conducted on categorical variables to assess the significant association between categorical features and the dependent variable. P<.05 was considered statistically significant.

Feature Engineering

Resampling

In this study, 3 resampling techniques, namely random undersampling, synthetic minority oversampling technique (SMOTE) oversampling, and SMOTE-Tomek combined sampling were experimentally compared during the dataset preprocessing phase. It was found that random undersampling performed best in improving the prediction accuracy for the minority class. Therefore, random undersampling was applied to the training set, where a portion of the majority class samples were randomly removed to reduce the majority class sample size. This approach helped mitigate model bias toward the majority class and improved overall classification performance.

Feature Processing and Algorithm Selection

To enhance model accuracy and efficiency, reduce computational costs, and avoid overfitting, feature selection was conducted. Before feature selection, Pearson correlation analysis was conducted for continuous features, while Kendall correlation analysis was used for discrete features. By examining the correlation matrix of the features, pairs of highly correlated features (with correlation coefficients >0.6) were identified, and perform dimensionality reduction on these highly relevant features. A total of 5 machine learning algorithms, including RF, LightGBM, XGBoost, GBDT, and CatBoost were used to construct predictive models on the training dataset. The model demonstrating the best performance across various evaluation metrics was selected from the baseline models (CatBoost was chosen based on experimental results). The CatBoost model was then trained, and the importance scores for each feature were determined. Features were added to the model in descending order of importance until the AUC score stabilized at its highest value with a specific number of features. At this point, no further features were added, and the model was constructed using the selected features.

Parameter Optimization and Model Evaluation

In addition to addressing class imbalance through resampling techniques, the model parameters were adjusted to assign different weights to each class, thereby balancing the class distribution. For the RF, GBDT, and CatBoost models, the parameter class_weight = “balanced” was set, while for the LightGBM and XGBoost models, scale_pos_weight = “ratio of majority and minority class” was used. Bayesian optimization was used for parameter tuning. Bayesian optimization is a global optimization algorithm based on Bayesian inference, which iteratively updates the posterior distribution of the parameters to identify the optimal configuration and optimize the objective function. To evaluate the optimization effects and improve the model’s generalization ability, 10-fold cross-validation was performed. The experiments were repeated 10 times, and metrics such as area under the receiver operating characteristic curve were averaged to assess model performance. In addition, model performance was evaluated using the F₁-score, accuracy, sensitivity, and specificity, while the receiver operating characteristic (ROC) curve and the area under the curve (AUC) were used to assess the model’s discriminatory ability. The Kappa value was also computed to evaluate the predictive capability of the model.

This research used several Python libraries and frameworks for data processing, feature engineering, model building, and hyperparameter tuning. The main tools used in the study are Python (version 3.6.4), NumPy (version 1.18.5), Pandas (version 1.1.5), SciPy (version 1.5.2), scikit-learn (version 0.24.2), LightGBM (version 4.3.0) XGBoost (version 1.5.2), CatBoost (version 1.1.1), and bayes_opt (version 1.4.0).

Feature Importance Ranking

Shapley Additive Explanations (SHAP) values are a method used to interpret machine learning model predictions. SHAP assigns a value to each feature, quantifying its contribution to the model’s output. Positive SHAP values indicate that the feature has a beneficial impact on the prediction, while negative SHAP values suggest a detrimental effect. The absolute magnitude of a SHAP value reflects the extent of the feature’s influence on the model’s decision. In this study, we used SHAP (version 0.45.0) for explainable machine learning to enhance interpretability.

Retrospective Cohort Validation of Model Application

Retrospective cohort data from a nondiabetic population were used as input to the aforementioned algorithms. The model outputs predicted the IR values for each participant, where a value of 1 was classified as high-risk IR and 0 as low-risk IR. Kaplan-Meier survival curves were used to conduct time-to-event analysis, comparing the 5-year diabetes incidence between the high-risk and low-risk IR groups. This approach further validated the model’s ability to accurately distinguish the onset of diabetes based on predicted IR levels.

Ethical Considerations

This study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of Xiangya Third Hospital, Central South University (approval no. 22206). All participants provided informed consent. The data were deidentified to ensure privacy protection.

Results

Characteristics of the Study Population

The study flow chart is presented in Figure 1. A total of 16,411 nondiabetic individuals who underwent health screenings were included in this study, with an average age of 42.74 (SD 11.36) years. The sample comprised 8205 males and 8206 females. Overall, 17% of participants were diagnosed with IR, with a higher prevalence observed in males compared with females. In the IR group, levels of weight, WC, BMI, SBP, UA, ALT, AST, TG, and TC were significantly elevated compared with the non-IR group. Conversely, the average HDL-C level was significantly lower in the IR group (P<.05). In addition, lifestyle factors such as diet, physical activity, and health literacy exhibited significant differences between the 2 groups (Tables 1 and 2).

‎

Feature Engineering

In the training set, random undersampling was applied to the majority class. Among the “physiological features,” 4 highly correlated features: “BMI,” “height,” “weight,” and “WC” were subjected to dimensionality reduction, with “BMI” retained. Similarly, for the 2 highly correlated features “SBP” and “DBP,” only “SBP” was retained.

Feature importance scores were computed for 55 lifestyle-related questionnaire features using the CatBoost algorithm. These features were ranked from high to low based on their importance scores and then sequentially input into the CatBoost model. After each addition, the model’s AUC score on the test set was recorded. As shown in Figure 2, the AUC score dropped significantly after adding 35 features, leading to the exclusion of 21 noncontributory questionnaire features. The 34 retained features, which showed significant contributions, include BMI, SBP, age, sex, family history medicine, exercise type, years of consistent exercise, exercise duration per session, weekly exercise frequency, dietary preferences, staple food structure, dietary taste, sleep duration, reasons for sleep disturbance, exposure to harmful substances, work physical demands, sedentary duration outside of work, consumption frequency of fruits, milk, meat, coffee, legume and soy products, sugary drink and fatty meats, daily vegetable intake, health check-up interval , self-monitoring of blood pressure and heart rate, observe urination and defecation, carrying emergency medications, regular meals, seat belt usage, bask, and awareness of normal salt intake and hazard ratio (HR).

In addition, the addition of 17 biochemical markers, BUN, Cr, UA, TC, TG, HDL-C, LDL-C, FBG, ALT, AST, TBIL, DBIL, TSP, ALB, GLO, A/G ratio, and TBA further improved the model’s accuracy, sensitivity, and specificity. Thus, the final feature selection for the model included the 34 lifestyle-related questionnaire features and 17 biochemical markers.

‎

Model Performance Evaluation and Feature Importance Ranking

Based on the 51 selected features (34 questionnaire features and 17 biochemical markers), 5 ML algorithms were applied, RF, LightGBM, XGBoost, GBDT, and CatBoost to build and evaluate models using both the training and testing datasets. The performance metrics of the 5 ML models are summarized in Table 3. The LightGBM model demonstrated the best performance across all metrics, including accuracy (0.7542), sensitivity (0.6639), specificity (0.7642), F₁-score (0.6748), and κ (0.3741). Although the AUC (0.8456) of the LightGBM model was slightly lower than that of CatBoost, LightGBM was determined to be the best model overall for predicting IR. The ROC curve for the best-performing LightGBM model is shown in Figure 3.

SHAP values were used to rank feature importance in the LightGBM model, as shown in Figure 4A. The top 10 features influencing IR were: BMI, FBG, HDL-C, TG, Cr, ALT, sex, TBIL, age, and A/G ratio. Figure 4B provides a visual representation of the SHAP values for the top 10 features, where BMI, FBG, TG, ALT, and sex were positively correlated with IR, while HDL-C, Cr, TBIL, age, and A/G ratio were negatively correlated with IR.

‎

‎

Retrospective Cohort for Validation and Clinical Significance Assessment

The retrospective cohort dataset was used to validate the differential occurrence of diabetes in populations stratified by the model. A total of 20,369 nondiabetic participants were continuously monitored (5101 individuals classified as IR high risk by LightGBM and 15,268 individuals classified as IR low risk), with the incidence of diabetes compared from baseline to 5 years. Out of 5101 high-risk IR participants, 235 (4.6%) developed diabetes, while 137 (0.9%) of 15,268 low-risk IR participants did. This resulted in a HR of 5.1, indicating a significantly higher risk for the IR high risk group compared with the IR Low Risk group. Figure 5A clearly illustrates the relationship between the cumulative incidence of diabetes in the high- and low-risk insulin resistance groups. The high-risk group exhibited a significantly higher cumulative incidence of diabetes compared with the low-risk group, particularly after the second year of follow-up, when the rate of increase accelerated. This suggests that individuals in the high-risk group have a higher likelihood of developing diabetes, and the cumulative effect of this risk becomes more pronounced over time.

The curve for the low-risk group starts at 1 (indicating that 100% of individuals were free of diabetes) and gradually declines over time, although the decrease is relatively small. In contrast, the high-risk group shows a much more rapid decline, especially after the second year, with a more noticeable downward trend (Figure 5B). The line chart depicting the diabetes incidence rates across different IR risk groups (Figure 5C) shows that the growth rate of incidence in the high-risk group is significantly higher than in the low-risk group. The incidence in the high-risk group rises sharply after the second year, reaching its peak by the fifth year. These findings suggest that individuals with high insulin resistance are more likely to develop diabetes within a shorter time frame.

‎

Discussion

Principal Findings

This study applied multiple machine learning methods using a health check-up database from southern China. Based on questionnaire data and easily accessible, low-cost laboratory measurements, a highly effective predictive model for IR was developed in a population with normal FBG. This ensures that the data required for model implementation is readily available, without the need for additional, expensive diagnostic procedures. This model holds significant value for precision screening of IR and for targeting interventions aimed at managing individual risk factors associated with IR. Compared with other models, LightGBM demonstrated a clear overall advantage in terms of its parameters. LightGBM is a gradient boosting framework that uses decision tree-based learning algorithms. It offers several benefits, including faster training efficiency, lower memory usage, and higher accuracy. In addition, it supports parallel learning and is capable of handling large-scale datasets [13].

Comparison With Previous Work

The previous studies have developed insulin resistance prediction models targeting different ethnic groups and populations, which proves that researchers recognize the necessity of conducting IR prediction. Tsai integrated the National Health and Nutrition Examination Survey public database in the United States and the MAJOR database in Taiwan, China, and established the HOMA-IR ML (XGBoost) prediction model for adults in the two regions. The AUC reached 0.87, the important feature spectrum revealed by SHAP analysis was similar to our results, confirming the determinative roles of factors such as BMI, FBG,TG, HDL-C, and sex [14]. Before this, the team also developed an IR model using samples from 1,229 chronic kidney disease patients sourced from the National Health and Nutrition Examination Survey database. The model included various nutritional and micronutrient indicators, and the overall evaluation metric showed that the XGBoost model achieved an AUC of 0.78 [15]. Zhang et al [16] developed a LightGBM model for IR in a cohort of nearly 10,000 adults aged 40 years or older from certain communities in Hubei, China, with an ROC score of 0.794. The most significant features included FBG, BMI, WC, TG, and sex. In a South Korean study involving 8842 residents, in addition to measurement and laboratory indicators, high-cost, high-quality features such as genetic risk scores, food frequency questionnaires, and nutrients were included, the best-performing model, XGBoost, achieved an AUC of 0.86 [17]. However, these studies either use expensive features or fail to address multicollinearity issues between features. For example, physiological knowledge and typical data support a strong correlation between BMI and waist circumference [18], but these studies did not account for multicollinearity, which could lead to overfitting.

In our preliminary model, using only low-cost questionnaire features did not yield the expected results. However, when laboratory indicators of liver function and blood lipids were added, the performance improved significantly. Both feature importance rankings based on LightGBM and SHAP value calculations showed that BMI, FBG, HDL-C, and TG were among the top 4 most important features. In fact, a 2003 consensus from the American College of Clinical Endocrinology [19] formally defined IR syndrome as a multisystem disease centered around IR, with key elements including IR or hyperinsulinemia, with or without related cardiovascular-endocrine metabolic abnormalities. These elements include overweight BMI, abnormal glucose tolerance, hypertension, elevated TG, or reduced HDL-C, based on the established epidemiological correlation between IR and these indicators [20,21]. From a pathophysiological perspective, IR and these metabolic indicators have an explainable causal relationship. For example, adipokines secreted by adipose tissue, such as adiponectin [22], tumor necrosis factor alpha [23] can modulate insulin sensitivity through the insulin signaling pathway. In addition, hydrolysis products of elevated triglycerides, such as free fatty acids, can induce insulin resistance by inhibiting insulin signal transduction and reducing the number of insulin receptors on target cells [24]. Low HDL-C improves insulin resistance through reverse cholesterol transport and anti-inflammatory effects [25]. We also found that liver function indicators, such as ALT, bilirubin, and the A/G ratio, play a significant role in IR prediction, with fatty liver being closely linked to IR [26]. These 3 abnormal indicators are common in individuals with fatty liver disease [27], which may indirectly reflect IR. In addition, kidney function indicators showed correlations, which could be related to factors such as physical inactivity, chronic inflammation, oxidative stress, vitamin deficiencies, adipose factor imbalances, and changes in the gut microbiota in populations with kidney damage [28]. Finally, we applied the best LightGBM model results to baseline IR stratification in a retrospective cohort, further validating the significant impact of IR prediction on diabetes incidence. This aims to clarify the practical value of such models. By comparing the incidence of diabetes in low-risk and high-risk IR groups, we observed that individuals at high risk for IR were more likely to develop diabetes in a shorter period, confirming the model’s value in assessing blood glucose metabolism.

Limitations and Future Directions

Despite the promising results in IR prediction, our study has several limitations. First, the data source is relatively limited in terms of geographical diversity. Our data came from a health check-up database from the Han Chinese population in Hunan Province in Central Southern China, which may impact the generalizability of the model to other regions or ethnic groups. Future studies could address this by incorporating data from different regions and ethnic populations to further validate the model’s generalizability. We would try to develop these ML models into user-friendly web pages or applications that are accessible to the general public and primary care providers, getting more input information and feedback to optimize our models, which is virtually a significant advantage of ML. Second, we tested the clinical implications of this trained model with a retrospective cohort, and have not yet explored the predictive efficiency of the model in prospective research, and we plan to perform it in a follow-up study in the near future. Third, the diagnosis of diabetes in the cohort was based only on the results of FBG, rather than oral glucose tolerance test or combined with postprandial blood glucose or glycated hemoglobin. However, we hope to base diagnoses on the latter as well in further research in the near future. Finally, although 5 ML algorithms were used in this study, there may be other algorithms with better performance that are currently available or soon to be developed. It would be necessary for us to further iterate models with more promising algorithms to improve the predictive performance of the models in the future.

Conclusions

In conclusion, the ML models using the LightGBM algorithm are efficient in predicting IR status in nondiabetic individuals. By leveraging low-cost laboratory indicators and questionnaire data, the model can accurately assess the current IR status in individuals with normal blood glucose levels, helping identify those at high risk of progressing to diabetes within large-scale populations.