The diagnostic process is a core task of general practitioners (GPs). However, making a diagnosis may be a challenging task given the diversity, complexity, and early presentation of symptoms. Clinical prediction models are intended to improve the diagnostic process [1]. These models can support the health care provider by predicting serious illness [2]. In the last years, the interest in artificial intelligence (AI) techniques for the development of prediction models has been growing [3,4]. AI-based prediction models could aid in decision-making for faster and more accurate diagnoses, with more diagnostic efficiency that can benefit patients’ health [5-8]. Examples are prediction tools that can predict colorectal cancer in patients [9,10].

Clinical prediction models used to be built on data from large databases, such as data collected for research purposes, claim data, or data from electronic health records (EHRs) [11,12]. EHR data consist of structured data, which are data in standardized format, and unstructured data, which are free-text data. Primary care (PC) EHR data provide extensive and longitudinal data from a patient’s health trajectory and changes over time. AI might prove to be a valuable method to extract clinically useful and actionable insight from this vast and complex source of patient data [13]. For that reason, AI has the potential to transform EHR data into a valuable tool for predicting diagnosis in daily PC practice.

Reviews on the value of AI in PC are scarce, and previous research had different aims. For example, Kueper et al [14] provided an overview of diagnostic prediction models based on AI in PC. However, the authors did not assess the quality of these diagnostic prediction models. Other research in this field explored AI systems in community-based primary health care [15] or focused on different machine learning (ML)–based diagnostic and prognostic models that predicted a health care condition [16]. As AI has the potential to support and improve the diagnostic process, high-quality and validated prediction models are crucial in order to ensure patient safety after clinical implementation. Although a variety of prediction models for PC have been developed, to our knowledge, there are currently no systematic reviews on AI-based diagnostic prediction models for PC using EHR data.

Evaluation of the content and quality assessment of AI-based diagnostic prediction models using EHRs in PC was largely lacking in current literature. Therefore, we systematically reviewed the literature in order to critically evaluate the content of these AI-based diagnostic prediction models, including risk of bias and applicability.

We performed a systematic review according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [17] (the PRISMA checklist is provided in Checklist 1). The protocol for this study was registered in PROSPERO (nr: CRD42022320002). The research team included stakeholders such as practicing GPs, researchers, methodologists, and AI experts in the design, analysis, and reporting of the study.

Our search was adapted from the search strategy developed by Kueper et al [14]. It combines two concepts including a wide range of different terms used to describe the concepts: (1) artificial intelligence and (2) primary care (for full search strategy, see Multimedia Appendix 1, part 1 [18-67]). EHRs were not part of the search strategy, because literature suggests that we might miss important studies when including EHRs or related terms in the search strategy [13]. We searched in the following databases: MEDLINE, Embase, Web of Science, and Cochrane. There were no restrictions concerning the publication date. The last search update was conducted on August 28, 2023. We focused on intervention and observational studies. We excluded systematic reviews, meta-analyses, case studies, editorials, protocols, and conference posters or abstracts. Full text had to be available to be selected for screening. The literature had to be written in English or Dutch. Duplicate publications were removed with EndNote 20.

Four inclusion criteria were used to select the papers: (1) primary care focus: this included PC data, models that were tested in a PC setting, or PC had to be specifically mentioned in the aim of the study; (2) diagnostic prediction model: models had to predict a health condition applicable during a GP’s consultation; prediction models that identified a disease in a database, rather than predicting a disease for an individual, were excluded; (3) AI: this included all ML and deep learning techniques; we directed our focus to data-driven prediction models without using medical images as input data; and (4) EHR-based data: EHR data had to be used for the development or validation of the model. EHRs were defined as PC data from EHRs, medical records, or clinical notes. See Multimedia Appendix 1, part 2 [18-67], for the full screening guidance.

Title and abstract screening was done in management software Rayyan (rayyan.ai) by 2 independent reviewers (LH and LvdH). Conflicts were resolved by a third reviewer (AU). Full-text screening was done by the same independent reviewers. Conflicts were resolved by discussion, and if no consensus was reached, they were resolved by a third reviewer (AU). Backward citation searching was conducted on the included papers and finished on November 7, 2023.

Data extraction of included papers was done by 2 independent reviewers (LH and AC). They used a standardized data extraction form adapted from the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) [68]. Basic information was extracted from all papers. The extraction of more detailed information was focused on EHR-based papers. For all papers (EHR and non-EHR papers), we extracted general information (first author, year of publication, title, data source, and country of data source), study design (retrospective or prospective), and outcome (predicted health condition). For the EHR papers, we additionally extracted dataset information (name of the dataset and sample size: number of participants used for model training, testing, or validation), AI technique, and predictors (the potentially used predictors used to develop the model). Risk of bias and applicability were assessed using PROBAST (Prediction Model Risk of Bias Assessment Tool). This tool includes 20 signaling questions divided into 4 domains (participants, predictors, outcome, and analysis) [69,70]. Overall judgment (ie, low, unclear, or high) of risk of bias is based on the 4 domains. If 1 domain is considered to have a high risk of bias, the overall judgment is scored as a high risk of bias. If at least 1 domain is considered to have an unclear risk of bias (without a domain with high risk of bias), the overall judgment is scored as unclear risk of bias. Applicability concern was rated based on 3 domains (participants, predictors, and outcome) and an overall judgment of applicability (ie, low, unclear, or high) was also given with the same approach as the risk-of-bias scoring. Applicability evaluation depends on the review question [69], and we translated applicability assessment as usability of the diagnostic prediction model in a PC setting. Conflicts in data extraction between the 2 reviewers (LH and AC) were resolved by discussion, and if no consensus was reached, they were resolved by a third reviewer (TvL).

We retrieved 10,657 records using our search strategy. After duplicate removal, we conducted title and abstract screening on 7146 records. A total of 347 records met the eligibility criteria for full-text screening. After full-text screening, 45 records were included. Backward citation searching yielded an additional 4 papers. A total of 49 papers were thus included in the review (Figure 1). Of the included papers, we identified 15 EHR papers and 34 non-EHR papers. A detailed description of the 34 non-EHR papers can be found in Multimedia Appendix 1, part 3 [18-67]. The data used in these 34 papers were collected from different sources, including secondary care datasets (n=17), questionnaires (n=4), and the knowledge of different health care providers (n=5).

Of the 15 EHR papers, 13 (87%) included the development of a prediction model [18-30]. In Table 1, the data extraction per paper can be found. The included EHR papers covered various outcomes, mostly chronic conditions (11/15, 73%) [19-28,31]. The most frequent predicted outcomes were dementia (3/15, 20%) [19,20,23], asthma (3/15, 20%), or chronic obstructive pulmonary disease (COPD) (3/15, 20%) [21,26,31]. Other study outcomes are shown in Table 1. All included papers used predictors registered in EHRs. Predictors included findings from clinical examination (n=6) [19,25,26,28,31,32], laboratory results (n=5) [21,22,25,28,32], and medication (n=4) [19,21,24,29]. All models used structured data.

Two papers externally validated and tested a prediction model in a PC setting [31,32]. One paper had a prospective approach and tested the diagnostic performance of a prediction model for asthma and COPD [31]. Ten papers (10/15, 67%) were published after 2020 [18,21-24,26,27,29,31,32]. Most data sources used in the studies originated from Europe (8/15, 53%) [18-22,24,27,31], followed by North America (5/15, 33%) [23,25,28,29,31,32] and Asia (2/15, 13%) [26,30].

All of the included studies performed at least 1 supervised AI technique (Table 1). The most used AI techniques were random forest (9 papers), logistic regression (7 papers), support vector machines (5 papers), boosting algorithms (5 papers), neural networks (5 papers), and naïve Bayes (4 papers).

None of the studies assessed by the PROBAST tool had a low risk of bias. We found a high risk of bias in 9 studies (9/15, 60%) and an unclear risk of bias in 6 studies (6/15, 40%; Table 2). In Multimedia Appendix 1, part 4 [18-67], the full assessment of the PROBAST tool can be found.

The most significant source of bias was found in the analysis domain. The main reasons for the high risk of bias in this domain were the insufficient number of participants with the outcome (5/15, 33%) [18,24,26,29,31] and irrelevant model performance measures that were used to evaluate the model (2/15, 13%) [28,31]. The main reasons for an unclear risk of bias in the analysis domain were lack of clarity on how missing data were handled (10/15, 67%) [18-20,22,23,27-30,32], and on how the predictors and their assigned weights in the final model correspond to results from the reported multivariate analysis (9/15, 60%) [18,20,21,25-30]. Although measures of calibration are not part of the signaling questions of the PROBAST, we noticed that only 4 papers (4/15, 27%) [22,23,29,32] used calibration to assess the performance of the model.

The second significant source of bias was found in the outcome domain. The main reasons for the high risk of bias in this domain were the determination of the predictors with a prior knowledge of the outcome (1/15, 7%) [31] and not excluding the predictors from the outcome definition (2/15, 13%). For example, Perveen et al [28] included fasting glucose levels to predict diabetes and Kocks et al [31] included spirometry findings to predict asthma and COPD. The 2 main reasons for an unclear risk of bias in the outcome domain were lack of clarity on the time interval between the outcome and the predictors (9/15, 60%) [20,24-30,32] and the lack of clarity on the outcome definition (7/15, 47%) [20-22,24,26,28,30].

The third domain with risk of bias was the participants domain. A high risk of bias in the participants domain was found because inclusion and exclusion criteria were not appropriate in 2 studies (2/15, 13%) as both studies excluded participants at high risk of the outcome [27,32]. Another reason for the high risk of bias was a nonappropriate data source that was used in 1 study [20] because the authors described the study as a case-control study although the study was not nested as recommended in the PROBAST guidelines [69,70]. The predictors domain was the domain with the lowest risk of bias. The lack of clarity that resulted in an unclear risk of bias covered mainly insufficient information on whether the predictors were defined and assessed in a similar way for all participants (4/15, 27%) [18,20,22,30].

Overall, we found an unclear concern for applicability in 10 papers (10/15, 67%) and a low concern for applicability in 5 papers (5/15, 33%; Table 3). The unclear concern for applicability to our research question was mainly noticed in the outcome domain due to a lack of clarity in reporting the time interval between the outcomes and predictors (8/15, 53%) [20,25-30,32].

In the predictors domain we also found an unclear concern for applicability due to the lack of clarity in the definition of the included predictors (5/15, 33%) [18,20,22,28,30]. For example, 1 paper lacked information on how notes were annotated before they were used as predictors in the model [18]. The unclear concern for applicability in the participants' domain was mainly due to lack of information on inclusion and exclusion criteria (3/15, 20%) [25,26,30].

We systematically reviewed the literature for studies about AI-based diagnostic prediction models for PC. These models were developed with different data sources, such as questionnaire data, secondary care data, or EHR data. Only 15 out of 49 models were developed using data from EHRs. Most of the models using EHR data focused on just 1 chronic condition. Merely 2 papers tested the model in a PC setting. All of the included studies performed at least 1 supervised AI technique, most often with random forest or logistic regression. Evaluation with the PROBAST guidelines showed an unclear to high risk of bias for all EHR papers. In most of the papers, we found unclear concerns about the applicability to our research question.

To the best of our knowledge, only 2 reviews evaluated the risk of bias in clinical prediction models on a wide range of diseases in PC studies [15,16]. Most of the included studies in these reviews showed a high to unclear risk of bias, which is in line with our findings [15,16]. However, there appear to be differences in grading compared with Abdulazeem et al [16]. They considered incomplete reporting and the absence of external validation a high risk of bias, whereas in our systematic review, these points were considered as an unclear risk of bias and no risk of bias, respectively. The study by Abbasgholizadeh et al [15] did not report details on the reasons they coded subdomains as high or unclear risk of bias, for which reason we are unable to make a formal comparison with our results.

Systematic reviews evaluating AI-based clinical prediction models in other medical fields have followed the same grading criteria as we did and found similar flaws in the analysis domain as we did in our systematic review [71,72]. These similarities include the unjustified small sample size in EHR studies, inappropriate evaluation in the performance measures, and flaws in handling of missing data [71-73].

The most used AI techniques were random forest, logistic regression, support vector machines, boosting algorithms, and neural networks. In previous systematic reviews, random forest and support vector machines are also more often found as most used methodology [16,71,73-75]. This might be explained by the well-described strong performance and ease of interpretability of random forests and support vector machines, particularly when working with lower-quality structured data. Most PC EHRs are primarily used for clinical purposes, with secondary purposes for research [69]. Thus, the challenges associated with using such EHRs to develop prediction models have been widely documented and include missing values and inconsistencies in data entry [13]. These challenges are inherent to the data and should be addressed at the preprocessing stage. We did not find papers that used generative AI methods (such as large language models). Our retrieved papers developed or validated tools based only on structured data (numbers or codes such as laboratory results, vital signs, and diagnosis codes from International Classification of Primary Care or ICD-10 [International Statistical Classification of Diseases, Tenth Revision]) rather than unstructured data or written text, where large language models work well on. Literature found it valuable for the performance of the model to use unstructured data together with structured data for prognostic prediction models [76,77]. We think that future studies about diagnostic prediction tools will increasingly use generative AI methods, although it is still difficult to integrate them into clinical workflows [78].

In general, studies analyzing EHRs are subject to a high risk of bias, because these data are collected for clinical rather than research purposes [69]. Hence, clinical prediction models developed on EHRs are more difficult to reproduce and generalize, given the heterogeneity of coding systems and database infrastructures [16]. In line with models analyzed in previous studies [15,16,73], most of the clinical prediction models were not externally validated. Most of the studies developed in PC were performed in high-income countries and may not have taken into account regional or global differences in the availability of certain predictors [14-16]. For example, some predictors may not be easy to obtain in PC settings in low-income countries (eg, spirometry results for the prediction of asthma or COPD). Furthermore, the lack of stratified analyses in most studies implies that we cannot draw conclusions about how diagnostic models perform across different equity groups. Together, all these factors limit the generalizability of the clinical prediction models.

The main strength of the study is the extensive search strategy with no date limit in a large and diverse range of studies on AI prediction models in PC. Not including “EHR” in the search strategy added rigor to our study as a recent review suggests that important papers could have been missed when we included EHRs in the search strategy [13]. A second strength is that the findings on the risk of bias were carefully assessed by 2 independent reviewers (LH and AC) with experience in clinical PC, and the conflicts were discussed with other experts in the field of PC and AI. Unlike previous systematic reviews that found a high proportion of studies with a high concern of applicability to the research question [72], we noticed no high concern for applicability in any study. We believe that the findings shared in our review are highly reliable in highlighting the current situation of AI studies in PC using EHRs.

The main limitation of this study is the broad definition of the terminology for the search strategy, which may have prevented us from capturing all relevant studies. For example, we included all studies that used ML and deep learning techniques. Given the lack of a widely accepted definition of AI, other reviews use other criteria for AI or ML [71,73,75]. Similarly, given our definition of diagnostic prediction models, we considered a diagnostic prediction model to be a model that predicts a health condition during a GP’s consultation. As a result, multiple prediction models that identified a disease in a database were excluded. The second limitation is the use of the PROBAST guidelines to determine the risk of bias and applicability in evaluating AI prediction models. Although the PROBAST guidelines are highly detailed and reliable in evaluating clinical prediction models [33], PROBAST has been criticized for being less specific and less applicable for AI-based models than traditional statistical methods. Considering this criticism, a protocol on the extension of PROBAST into PROBAST-Artificial Intelligence (PROBAST-AI) has been published with the aim to develop a PROBAST-AI tool to better support evaluation of prediction model studies that applied AI [3]. The PROBAST-AI tool has not yet been published.

The relevance of the applicability of prediction models in clinical practice should be the priority when developing clinical prediction models, as stated in a number of standardized frameworks designed for prediction model developers [79,80]. We found that only 2 models were tested in PC settings. Moreover, most studies included in this review predict chronic conditions. This is also seen in previous reviews evaluating clinical prediction models in PC [14,16]. However, in general, chronic conditions are not known to be difficult to diagnose in PC. Two examples from our included papers are the diagnosis of hypertension predicted on the variable high blood pressure [25] and the diagnosis of diabetes predicted on the variable high glucose levels [28]. These predictions might not be as useful in clinical practice, even if the model performance metrics are excellent. Nevertheless, chronic conditions are highly prevalent in PC and for conditions that are influenced by several and complex factors, prediction models may facilitate the diagnostic process for the GP. As most tools focused on predicting 1 condition, GPs would have to use many prediction tools side by side to predict the correct diagnosis in daily practice. All these findings highlight that involving more practicing GPs and asking what they need are important in developing clinical prediction models with a higher success rate of clinical implementation. We recommend involving relevant stakeholders in the early stages of the development of a new model.

To improve the methodology in future studies, our findings suggest that a special focus is required on reporting areas such as methods for internal validation, appropriate inclusion of participants, and a proper sample size calculation. A high risk of bias mainly found in the analysis and outcome domains should be alarming as this questions the methodology of the included papers. We found an unclear risk of bias and unclear concern for applicability in more than half of the included studies, mainly related to poor reporting, for example, about missing data. Missing data is known as a large challenge for EHR data [13], and extra attention should therefore be paid to reporting this. Researchers can benefit from the use of the TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) statement [81] and PROBAST guidelines in communicating their findings [3], particularly now that the TRIPOD-AI extension is released [82]. To enhance the applicability of the prediction model, we highlight the importance of clear reporting on the time interval between predictors and outcome, a clear definition of the outcome and predictors, and a clear description of the inclusion and exclusion criteria. Differences in recording between EHRs might lower the performance of the model in the external validation step, and external validation is a crucial step for generalizable and reliable models [76]. However, we found only 2 papers that performed external validation.

AI-based prediction models using EHR data are not yet ready for implementation into PC daily practice. The number of studies found was limited, and reproducibility and generalizability were insufficient. For a diagnostic prediction model to be used in PC, it is important that GPs and relevant stakeholders are involved in the development, that the model is externally validated, and that it is appropriately recorded.