This study was submitted to PROSPERO (International Prospective Register of Systematic Reviews; ID: CRD420250656632) and adhered to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.

The inclusion and exclusion criteria for study selection are summarized in Textbox 1.

We systematically searched 4 databases (Web of Science, Cochrane Library, Embase, and PubMed) from inception to 1 January 2025. The search strategy involved a combination of Medical Subject Heading terms and free-text words. Detailed search strategies are provided in Table S1 in Multimedia Appendix 1 [10,13-28]. The search had no regional restrictions.

The literature retrieved from the searched databases was imported into EndNote software (Clarivate). After eliminating redundant entries, the remaining articles were systematically screened based on titles and abstracts to discard references that did not meet the predetermined eligibility requirements. The full texts of the remnant studies were screened to obtain additional relevant studies.

Data extracted included titles, year of publication, first author, study type, country, patient sources, study design, treatment regimen, outcome definitions, radiomics source, segmentation method, number of imaging investigators, ROI segmentation software, number of cases and total number of outcome events in the validation and training sets, type of model used, variable screening method, modeling variables, overfitting assessment, and model rating metrics.

Two authors (YS and BL) independently selected the studies and extracted the data, with a third researcher (JA) consulted to resolve any disputes. The interrater agreement between the 2 independent reviewers during the study selection process was excellent, with a Cohen κ value of 0.895.

This meta-analysis included studies that used radiomics-based ML models to predict CRC recurrence risk. The Radiomics Quality Score (RQS) was used to determine the reporting completeness and methodological robustness of the included articles [29]. This 16-item scoring system, with a maximum score of 36, is specifically designed to facilitate radiomics studies that encompass various aspects, including image protocol quality, multiple segmentation processes, phantom studies across all scanners, imaging conducted at multiple time points, feature reduction techniques, and multivariable analyses incorporating nonradiomics features. In addition, it addresses the detection and discussion of biological correlates, cut-off analyses, discrimination statistics, calibration statistics, and prospective studies registered in trial databases. Furthermore, the system emphasizes the importance of validation, comparison against established “gold standards,” potential clinical use, cost-effectiveness analyses, and adherence to principles of open science and data sharing. Existing radiological studies are challenging to evaluate within the RQS framework, as they often lack phantom studies across different scanners, imaging at multiple time points, identification and discussion of biological correlates, prospective trial registration, and cost-effectiveness analyses.

Two investigators (YS and CJ) administered the RQS measure and cross-checked the results upon completion. A third investigator (HS) was consulted to resolve any disputes.

This meta-analysis aimed to synthesize the discrimination metrics (c-index) for evaluating the overall accuracy of ML models. For primary studies with 95% CI values or SEs for c-index values, the standard errors were estimated using Debray et al [30] methodological framework. Heterogeneity across studies was evaluated quantitatively using the I² index. The restricted maximum likelihood method was used to estimate between-group variance, which is recommended for its improved performance in handling heterogeneity, particularly when the number of studies is limited [31]. Given significant heterogeneity (defined as I²>50%), a random-effects model was adopted for meta-analysis, and sensitivity analyses were performed. Publication bias was assessed through funnel plot asymmetry analysis and Egger test. Subgroup analyses were conducted to further detect possible sources of heterogeneity. Subgroup analyses were conducted based on imaging modality (CT vs MRI) and dataset type (training vs validation sets). The meta-analysis was conducted in Stata 15 (StataCorp LLC). In addition, to calculate the 95% prediction intervals, which estimate the range within which the true effect of a future study would be expected to fall, we used R software (version 4.4.3; R Development Core Team).

The GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) approach was used to determine the overall certainty of evidence for our primary outcome (predictive performance measured by c-index) [32]. Given that the included studies were observational in design, the initial evidence level was low. Therefore, we evaluated the evidence across the 5 GRADE domains (risk of bias [RoB], inconsistency, indirectness, imprecision, and publication bias), creating a final certainty rating for each model comparison.

The initial search of the 4 databases yielded 5916 articles, of which 5049 remained after removing duplicates. Two reviewers independently assessed the title and abstract of all remaining articles, of which 5007 were excluded due to inconsistencies with the study goals. Reasons for exclusion were primarily nonrelevant study type (eg, reviews), not focusing on CRC radiomics or ML, or not having recurrence as a prediction outcome. The remaining articles were further subjected to a careful full-text review, after which 25 articles (5 conference abstracts, 12 articles without CRC recurrence endpoints, and 8 articles missing indicators for assessing model accuracy) were removed. Therefore, only 17 articles [10,13-28] were included in the study’s analyses. Figure 1 shows the study selection process.

This meta-analysis included 17 studies (involving 4600 patients with CRC) [10,13-28], which were published between 2019 and 2025. All 17 studies were cohort investigations, of which 12 were conducted in China [10,13-15,17,18,20,21,23,25-27], and the rest were conducted in Canada [16], France [24], South Korea [28], the Netherlands [19], and the United States [22]. Furthermore, 8 [16,20,22,23,25-28] and 9 [10,13-15,17-19,21,24] studies were single- and multicenter investigations, respectively. In addition, 7 [16-18,20,23-25], 9 [10,13-15,19,21,22,26,28], and 1 [27] studies involved patients with CRC, rectal cancer, and colon cancer, respectively. The included studies encompassed patients from stage I to IV, with the majority involving stages II and III. Most patients underwent curative-intent surgery, and 2 [18,20] studies specifically included cases with resectable metastases. All studies conducted radiomics analysis, of which 8 [13-15,19,21,22,26,28] and 9 [10,16-18,20,23-25,27] were based on MRI and CT images, respectively. Only 1 included study applied a DL model [16], while the remaining used diverse traditional ML algorithms [10,13-15,17-28]. Regarding the number of researchers involved in image segmentation, 2 studies [24,28] had only 1 imaging researcher, 9 studies [10,13,14,17-21,23] had 2 researchers working together, 4 studies [15,22,25,26] had 3 researchers working together, and 2 studies [16,27] did not report the number of imaging researchers. Regarding the segmentation software used for ROI regions, ITK-SNAP and 3D Slicer were used in 8 [10,13-15,17,18,20,21] and 3 studies [20,24,25], respectively, with the rest of the studies using The Medical Imaging Interaction Toolkit (MITK) [16], Gold LX [22], INFINITE PACS [23], Radcloud radiomics platform, and the Eclipse system. Two studies [16,27] did not perform validation set partitioning, 1 study [22] performed 5-fold cross-validation, and 8 studies [10,13-15,17-21,25] had a separate external validation cohort [10,13-15,17-19,21], of which 5 studies [13-15,17,18] had 2 cohorts (internal and external). In 14 studies [10,13-21,23-25,28], clinical factors were combined with radiomics features to construct models, while the rest of the studies developed models using radiomics features or clinical factors alone [22,26,27]. Table S2 in Multimedia Appendix 1 [10,13-28] shows the basic characteristics of the included studies. Table S4 in Multimedia Appendix 1 [10,13-28] shows essential data for pooled analysis.

The average and median RQS scores of all 17 studies were 13.23 (SD 2.56) and 13 (IQR 6-16), respectively. All studies had “Image protocol quality,” “Discrimination statistics,” and “Cut-off analyses” items. In addition, 16 [10,14-28] out of 17 studies (94%) had “Calibration statistics” and “Validation” items [10,13-26,28]. On the other hand, 13 [10,13-15,17-23,25,26] out of 17 studies (77%) conducted “Multiple segmentations.” Fourteen [10,13-21,23-25,27,28] out of 17 studies (82%) performed multivariable analysis and incorporated nonradiomics features [10,13,14,16-18,21-28], potentially yielding a more holistic model. A total of 11 [10,13,15,17-21,26-28] out of 17 studies (65%) reported potential clinical use and generated clinical decision curves. Two [15,16] studies provided open science and data access, and only 1 study [23] detected and discussed biological correlations. There were no phantom studies, and none of the studies used the comparison to the “gold standard,” adjustment for multiple testing or feature reduction, prospective designs, imaging at multiple time points, or cost-effectiveness analysis. Supplementary material provides the detailed RQS scores for all included studies (Table S3 in Multimedia Appendix 1) [10,13-28].

The RoB and concerns regarding applicability for each included study were rigorously assessed using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool [33]. Notably, the included studies were cohort studies, and most of them were not excluded, which avoided the low RoB in case selection. Furthermore, considering that the included studies primarily used supervised ML with clearly defined outcomes, they had a low RoB. The implementation and interpretation of gold standards were performed by using clear criteria for recurrence, and the blinding did not influence outcomes; hence, the RoB was minimal. In addition, there was low RoB in case selection, given that known gold standards were applied. However, given that both studies enrolled cases of CRC-related metastases, there was a high RoB in the clinical applicability. A traffic-light plot summarizing the QUADAS-2 assessment for each study is presented in Figure 2.

This meta-analysis systematically evaluated and compared the predictive performance of different radiomics-based ML models for CRC recurrence risk. We postulated that models integrating both radiomic features and clinical variables may show superior predictive accuracy compared to either modality alone. The results revealed that the integrated models achieved the highest pooled c-index of 0.83 (95% CI 0.79‐0.87) in the validation set, outperforming models based solely on radiomics features (c-index: 0.80; 95% CI 0.75‐0.85) or clinical features (c-index: 0.73; 95% CI 0.68‐0.79). Collectively, the findings of this study underscore the complementary value of combining quantitative imaging biomarkers with clinical data to enhance recurrence risk stratification in patients with CRC.

Notably, research on cancer diagnosis has seen a proliferation of systematic reviews and meta-analyses, particularly concerning CRC diagnosis, treatment response, and prognosis [34,35]. While this reflects the high clinical interest in radiomics, it also underscores the challenge of demonstrating novel and impactful contributions. Our study addresses a specific and clinically pertinent niche within this saturated field. Unlike previous reviews that often focus on a broader diagnostic or prognostic scope, this meta-analysis provides a dedicated quantitative synthesis focused exclusively on ML-based models for recurrence risk prediction. More importantly, we placed a particular emphasis on evaluating the incremental value of integrating radiomics with clinical features, a comparative approach that is less commonly the central focus of existing reviews. Therefore, we believe this work offers a nuanced and actionable insight: that the future of radiomics in clinical decision-making may lie not in replacing clinical factors, but in synergistically combining with them. This promising finding, however, should be considered in light of the detected publication bias, which indicates that the literature may lack smaller studies with negative results. Therefore, there is an urgent need for prospective validation studies in the future to confirm these promising findings.

Multiple reviews and meta-analyses have reported the clinical use of potential biomarkers in predicting CRC recurrence postsurgery [36,37]. Nonetheless, the predictive value of radiomics remains inadequately assessed. Several studies have constructed different clinical prediction models for CRC recurrence risk. For instance, Alinia constructed a predictive model and validated its predictive efficacy using 7 ML algorithms based solely on clinicopathological features (eg, disease stage and treatment type) in a cohort of 284 patients with CRC [38]. In addition, Mohammadian Rad predicted recurrence risk using a gradient-boosting model that combined clinicopathological features (eg, carcinoembryonic antigen level) and demographic data [39]. Furthermore, Seong et al [40] integrated unstructured textual information from colonoscopy reports with clinical data for CRC risk stratification.

Despite their good predictive results, these studies had some limitations, which could be attributed to several factors. First, the performance of the models based on clinical data alone in predicting CRC recurrence risk was lower than that of the fusion model integrating radiomics features, highlighting the significance of combining imaging features for an improved predictive ability. Second, colonoscopy is highly invasive and relies largely on the endoscopist’s experience and skill level; hence, its images are mainly based on morphological features, which lack quantitative analysis of deep infiltration or the tumor’s metastatic potential, thus limiting the model’s clinical applicability. Conversely, radiomics can extract hundreds of quantitative features (eg, texture and grayscale covariance matrix) from CT and MRI, among other images, reflecting tumor heterogeneity more comprehensively and providing a more reliable individualized treatment basis.

Herein, the CT and MRI images were predominantly used to predict CRC recurrence risk. Notably, there were no significant differences between the CT and MRI radiomics-based prediction models in the validation set. Nonetheless, MRI radiomics studies often require integrated multisequence image segmentation and feature extraction, whereas CT radiomics is usually based on a single modality (eg, enhanced CT) and features a relatively simplified process. In other words, MRI image segmentation has a greater workload [41]. Given that there is no significant difference in the predictive value between the 2, CT-based radiomics, despite its simplicity, can still be considered in the future in constructing radiomics-based ML prediction models.

Selecting ML algorithms remains a notable challenge when constructing radiomics-based predictive models, especially for image-processing tasks. Notably, ML models could be influenced by traditional ML and DL algorithms [42]. Traditional ML relies on the (manual) segmentation of images, image texture screening for model construction, and model validation. When extracting and screening image textures, some of the image information may be lost, somewhat impacting the model’s accuracy [43]. On the other hand, DL can intelligently segment images or be trained directly on segmented images. Furthermore, it incorporates the extraction and screening of image texture features into the training process, maximizing image information retention and providing better accuracy [44,45]. Here, manual segmentation was used owing to the few enrolled studies. Therefore, future studies should further explore the application of DL methods in constructing radiomics-based predictive models, potentially improving image recognition.

The role of clinical and demographic characteristics is particularly important when constructing prediction models based on radiomics, as they can accurately reflect disease progression. Some studies reported a significant correlation between laboratory test results and tumor staging and prognosis information, including tumor markers [46,47]. In addition, social factors such as marital status and family income were markedly linked to the prognosis of patients with tumors [48,49]. These studies highlight the potential significance of screening and incorporating meaningful clinical and sociodemographic characteristics when constructing radiomics-based prediction models. To effectively integrate these multimodal models into clinical practice, future efforts should focus on developing interoperable digital health platforms that can seamlessly combine radiomic features with real-time clinical and demographic data. Such systems should be designed to interface with existing electronic health records, support automated image analysis, and generate interpretable risk scores that can be readily used by clinicians for personalized recurrence risk assessment. In addition, implementation studies are needed to evaluate the usability, workflow integration, and clinical impact of these tools in routine care settings.

When constructing a clinical prediction model, validation is often conducted both internally and externally. For internal validation, both the training and validation sets are often derived from the same dataset and split through random sampling, among other methods [50]. Notably, internal validation often has limitations on the model’s interpretation and generalizability, especially in radiomics research that is highly dependent on images. Consequently, external validation, which involves datasets from different sources, is often recommended [51]. Herein, internal and external validation performances were assessed separately, revealing that the latter exhibited desirable prediction accuracy.

The significant statistical heterogeneity observed in our meta-analysis, while addressed by the use of a random-effects model, warrants a deeper qualitative discussion regarding its potential sources. The methodological diversity across the included studies likely introduced substantial variability that influences the interpretation of our pooled estimates. For instance, differences in imaging protocols (eg, CT vs MRI, scanner manufacturers, and contrast-enhancement phases) directly affect radiomics feature values, making it challenging to harmonize models across studies. Variations in patient characteristics, such as the inclusion of different cancer stages (I-IV), alter the underlying recurrence risk profiles of the cohorts. Simpler models, such as logistic regression, offer high interpretability but may fail to capture intricate, nonlinear relationships in the radiomics data. In contrast, more complex traditional ML algorithms (eg, random forest and support vector machines) can model these nonlinearities and often achieve higher accuracy, albeit at the cost of increased computational demand and potential overfitting if not properly regularized. While DL models hold the promise of end-to-end feature learning and potentially superior performance by automatically discovering relevant patterns from image data, they were scarcely represented in our included studies and require large datasets to train effectively. Finally, the definition of the recurrence outcome itself varied, encompassing disease-free survival, local recurrence, or time-to-recurrence, each capturing a slightly different clinical endpoint. Despite the methodological heterogeneity, our meta-analysis offers a comprehensive and robust synthesis of the current evidence. The fact that a consistently strong discriminative performance (c-index >0.80) was maintained across such varied technical and clinical contexts is a key finding, underscoring the robustness of radiomics-based prediction for CRC recurrence.

Despite its valuable insights, this study had some limitations. First, the number of eligible studies was limited, which constrained more granular subgroup analyses. In addition, the geographic origin of the evidence base was imbalanced, with 12 of the 17 included studies conducted in China. While this provides a robust assessment within that specific context, it may limit the generalizability of our findings to other populations with different genetic backgrounds, ethnicities, and healthcare systems (eg, in terms of screening protocols, treatment strategies, and staging criteria). Therefore, further multinational studies are advocated to validate the broader applicability of these radiomics models and to investigate potential geographic or ethnic variations in their performance. Second, the limited number of studies resulted in even less data for the subgroup analysis of images, possibly affecting result interpretation. Third, although subgroup analyses based on the image source (CT vs MRI) were performed in the validation set, thereby explaining some of the heterogeneity, there was significant heterogeneity. This likely reflects the clinical and methodological diversity across studies, including differences in imaging protocols, patient characteristics, model types, clinical variables, and recurrence definitions. These factors, compounded by the limited number of studies, make it difficult to quantitatively identify other potential sources of heterogeneity and highlight the challenges in achieving standardized validation across independent cohorts. Furthermore, regarding reproducibility, only 2 studies provided open-source code or datasets. This lack of transparency prevents independent validation of the proposed models and limits the clinical translation of our findings. In the future, researchers should promote open science practices by sharing code and data where possible to facilitate verification and build upon existing work. Finally, the assessment of model performance was primarily based on the c-index due to inconsistent reporting of calibration metrics (eg, Brier score and calibration slope) and time-dependent discrimination measures (eg, time-dependent–area under the curve) across studies. While the c-index provides valuable evidence of the models’ ability to stratify risk, this assessment should be complemented by future evaluations of calibration to ensure the accuracy of predicted probabilities for individual patients. Establishing robust calibration will be a crucial next step in translating these promising discriminative models into reliable clinical tools. These aspects represent important limitations that should be addressed in future studies with larger and more standardized datasets.

This study demonstrates that ML models based on radiomics and incorporating clinical features exhibit superior performance in predicting the risk of CRC recurrence, with a significantly higher discriminative ability (c-index) than models relying only on a single data source. This finding highlights the significant value of multimodal data fusion in improving prediction accuracy. However, most of the existing studies use traditional ML methods that rely on manual feature extraction and screening, which may lead to information loss and limited model generalization ability. Future studies should further explore the potential of end-to-end feature learning methods, such as DL, to improve model robustness and clinical translational value by automatically extracting high-level image features and reducing manual intervention, and prioritize prospective, multicenter validation with standardized protocols and explainable AI to facilitate clinical adoption.