The study included 272 patients with PeAF (aged 20-90 years) who were admitted for an initial CA at the Japanese Red Cross Kyoto Daini Hospital between January 1, 2015, and December 31, 2023, with available ECGs and patient background, echocardiography, and laboratory data within 3 months. A total of 189 (69.5%) patients were included in the study, excluding 14 (5.1%) patients whose ECG recorded a pacemaker rhythm, 30 (11%) whose QRS waves were difficult to exclude or whose extracted f waves were too short for a feature analysis, and 39 (14.3%) whose AF duration was difficult to determine.

The following methods were used to define PeAF (AF duration of less than 1 year) and LsPeAF (AF duration of 1 year or more). First, PeAF was defined if any of the following conditions were met: (1) sinus rhythm was recorded on the ECG within 1 year, excluding postdefibrillation ECGs, followed by a consecutive recording of AF; and (2) the onset of clinical symptoms or an irregular pulse clearly related to AF was confirmed within 1 year, followed by a consecutive ECG recording of AF. Second, LsPeAF was defined as any of the following: (1) patients with ECG recordings of AF confirmed for more than 1 year consecutively and (2) patients with onset of clinical symptoms or an irregular pulse clearly related to AF more than 1 year prior, followed by consecutive ECG recordings of AF within 1 year. Cases that did not meet any of the above conditions were excluded as cases with an unknown duration.

The ML model was developed and cross-validated to determine whether the duration of AF was more than 1 year, using 145 patients with a registration period from January 1, 2015, to March 31, 2023, as training data. The predictive ability of the ML model was tested using 44 patients whose registration period was from April 1, 2023, to December 31, 2023, as test data (Figure 1).

In phase 1, a total of 10 cardiologists (group A) evaluated the test data of 44 patients using their ECGs and patient information and answered whether each patient had an AF of at least a 1-year duration. In phase 2, they answered that again after being presented with the predictive results of the ML model and the Shapley additive explanation (SHAP) values, which calculated the contribution of each variable feature to the predictive results of the model. The case presentation in phases 1 and 2 is shown in Figure 2. Following that, group B was shown with the diagnostic results of group A, and the same 2-phase tests were conducted as in group A, with a prior recognition of the cardiologists’ ability to diagnose the AF duration. Finally, all cardiologists were asked whether they attached any importance to each of the characteristics as a decision-making factor.

Continuous variables are presented as the mean (SD). Categorical variables are presented as frequencies (percentages). Statistical significance was determined using the Mann-Whitney U test for continuous variables and the chi-square test for categorical variables; a P value <.05 was considered statistically significant. Python software (version 3.10.12; Python Software Foundation) was used for statistical analysis.

The percentage of correct answers for each cardiologist was calculated from the results of the cardiologist’s test and compared between phase 1 and phase 2 in group A and group B, respectively, using a paired sample 2-tailed t test.

The study was approved by the Medical Ethics Review Committee of the Kyoto Prefectural University of Medicine (ERB-C-2195-1). Informed consent was given on an opt-out basis, and all data were anonymized. The study was conducted without offering any financial or material rewards to the participants.

Of the 189 patients studied, 130 (68.8%) had PeAF and 59 (31.2%) had LsPeAF (Table 1). There was no significant difference in age (P=.50), and in terms of sex, there were significantly more male patients in the LsPeAF group (P=.01). There were no significant differences in the medical history between the 2 groups (all P>.05). On examination, echocardiography showed no significant difference in LAD between the two groups (P=.11). The ECG data showed that the HR was significantly lower in the LsPeAF group (P<.001), but there were no significant differences in the QRS wave width or amplitude (all P>.05).

Among the f wave characteristics, the aVR and V1 RMS were significantly lower in the LsPeAF group (P=.05 and P=.04, respectively), and the other inductions tended to have lower mean values overall, but there were no significant differences (all P>.05). No significant differences were observed in the DF and OI in any of the inductions (all P>.05; Table 2).

In this study, model 4, trained with XGBoost, which had the highest AUC among the ML models created, was adopted (Multimedia Appendices 2 and 3). To evaluate the prediction accuracy of the ML model, a random seed was set 5 times, and the mean receiver operating characteristic–AUC curve was calculated; the mean AUC of those 5 was 0.82 (Figure 4).

In the SHAP value, the ML model gave the highest importance to HR. The following weightings were then assigned in the following order: RMS, SampEn, amplitude, left ventricular ejection fraction, sex, OI, BMI, QRS wave amplitude, QRS width, LAD, and age (Figure 5).

The predicted results for the test data were 0.82 for accuracy, 0.72 for sensitivity, 0.81 for specificity, 0.81 for positive predictive value, 0.82 for negative predictive value, and 0.77 for F₁-score.

The diagnostic accuracy of the AF duration by cardiologists is shown in Figure 6. The mean percentage of correct answers in group A was 63.9% (SD 9.6%) for phase 1 and improved to 71.6% (SD 9.3%) for phase 2 (P=.01). The mean percentage of correct responses in group B was 59.8% (SD 5.3%) for phase 1 and improved to 68.2% (SD 5.9%) for phase 2 (P=.007). There was no difference in the percentage of correct answers between groups A and B for phase 2 (P=.48). The diagnostic accuracy with the cardiologists significantly improved with ML model support in both groups. The mean percentage of answers that differed from the predictions of the ML model for phase 2 (percentage of answers where cardiologists did not trust the ML model and believed their own determination) was 17.3% (SD 10.3%) in group A and 20.9% (SD 5%) in group B, and there was no significant difference between groups A and B (P=.85).

When a posttest questionnaire was conducted to find out which characteristics were important to the 20 cardiologists, 18 (90%) said that the LAD was important. This was followed by 17 (85%) cardiologists selecting the f wave amplitude in the 12-lead ECG, 16 (80%) selecting age, and 15 (75%) selecting the HR in the 12-lead ECG (Table 3).

Regarding the prediction of AF duration, the goal was to achieve a more accurate prediction by using ML models. As a result, the part that had previously relied on the cardiologists’ experience and subjectivity was supplemented, and the uncertainty related to the prediction was reduced. Next, we adopted an approach that combined the cardiologists’ diagnostic capability with the ML models. This enabled the objective prediction with the ML model to be referenced against the cardiologists’ experience and subjective judgment, thereby improving the overall quality of the cardiologists’ diagnostic process.

There have been few reports that have validated the prediction of the duration using f wave characteristics: the AUC for the prediction using the amplitude of the f wave in lead II of the 12-lead ECG was 0.77, the AUC for the prediction using the DF of the f wave in lead V1 was 0.63 [14], and the AUC for the prediction using f wave learning with the ML model was 0.78 [15]. The ML model developed in this study was trained not only with the ECG characteristics including the f wave but also a past history and echocardiographic and laboratory data, which could improve the prediction accuracy to AUC 0.82 despite the small dataset. A validation using the test data also confirmed the result of an accuracy of 0.818. Therefore, in both groups A and B, the support of the ML model improved the ability of the cardiologists to diagnose the AF duration.

On the other hand, group A had a different answer selected for 24.3% of the ECGs compared to the ML model. One possible reason for that was likely to be that they believed their own decision over the ML support, as they did not know the superiority or inferiority of the cardiologists and ML model in predicting the duration of AF.

Therefore, an additional test was conducted with group B. The accuracy of the cardiologists’ diagnosis (percentage of correct answers in group A) was shown to group B in advance so that they were aware of the superiority or inferiority between the accuracy of the cardiologists’ diagnosis and the accuracy of the ML model. The same diagnostic test was then conducted with group B as in group A. Surprisingly, the percentage of cases selected that differed from the prediction of the ML model (21%) did not improve. That may have been due to the feature set of the model.

Among the SHAP values, the HR was the most correlated with the AF duration. Those facts are empirically recognized by cardiologists. In fact, 75% (15/20) of cardiologists consider HR to be important. For example, animal studies have shown that significant functional electrophysiological remodeling of the atrioventricular node occurs after prolonged AF, resulting in a spontaneous decrease in the ventricular HR [18]. Aging has also been shown to correlate with the atrioventricular node dysfunction as a result of atrioventricular node remodeling and ion channel expression [19]. In addition, patients with LsPeAF are often treated with drugs that control the HR, such as beta-blockers, calcium channel blockers, and digitalis. Those drugs decrease the HR and prevent it from becoming excessively fast. Taking those drugs over a long period of time may decrease the HR due to their cumulative effect. The dataset used in this study consisted of patients with AF scheduled for CA, and patients with a particularly large LAD tended to be excluded from the dataset at the discretion of their cardiologists because of their high risk of recurrence after CA. That selection bias may have been responsible for the discrepancy between the LAD characteristics and the ML prediction and cardiologist decision-making. The LAD can also be magnified by factors other than left atrial remodeling due to AF; Tsang et al [20] reported that left atrial enlargement is associated with congestive heart failure, vascular disease, transient ischemic attacks or strokes, and a history of smoking and is particularly strongly associated with diastolic dysfunction. Even in patients without a history of atrial arrhythmias or valvular disease, left atrial enlargement has been shown to increase with the severity of diastolic dysfunction, and remodeling due to AF (ie, LsPeAF) is not necessarily correlated with left atrial enlargement.

The lack of a difference in the diagnostic improvement between groups A and B indicated that cardiologists tend to trust their own experience when there is a discrepancy between their perception and the results of the ML model. That may be the case unless, for example, their judgment was clearly erroneous, in which case simply recognizing that the accuracy of the ML model was higher than the cardiologist’s diagnostic accuracy may not have been sufficient to influence their decision-making. Lee [21] noted that artificial intelligence (AI) and human predictive performance may differ depending on the type of patient, and a deeper understanding of AI and human predictive capabilities, respectively, will increase human reliance on AI. Therefore, for clinicians to trust AI, understanding why the algorithm came to the conclusion it did is important for them to trust AI. In this regard, in addition to the diagnostic explainability, one should also consider expressing predictions as probabilities. That is because greater clarity about the uncertainty of predictions may help cardiologists assess the risk and reliability of a diagnosis. For example, according to a study by Heuer and Breiter [22], users of ML models value not only the accuracy of the prediction but also the probability representation of the prediction. In other words, they may need a process that allows them to decide whether to perform additional tests or treatments based on the reasons for the prediction (visual presentation of the impact of each factor) and its probability.

This study had some limitations. First, the quality of the echo and ECG data and the problem of missing data may have affected the prediction accuracy of the ML model. Therefore, data cleaning and filtering were performed to exclude low-quality data and proceed with the analysis. As a result, the model’s prediction accuracy was 81.8%, with a sensitivity of 72% and specificity of 89%, thus confirming a certain degree of robustness. Second, this was a retrospective study and there may be some patients for whom the determination of the AF duration was uncertain. The determination of whether the duration was more than 1 year was based not only on interviews with the patients but also on physical examinations by cardiologists and ECG recordings. Furthermore, unclear symptoms and cases in which the timing was difficult to determine were excluded from this study, so we believe that the reliability and accuracy of the duration determination were high. Third, the study was not validated at other sites and the generalizability of the results to other sites and different populations was not confirmed. Also, the relatively small size of the dataset and the possibility of overfitting may have limited the robustness and generalizability of the model. Therefore, overfitting was avoided by “removing as many irrelevant variables as possible,” “cross-validation,” “validation with multiple seed values,” and “validation with test data from a different time period than the training data.” Those results confirmed that, even with the current dataset, the ML model complemented the cardiologists’ diagnostic ability and provided effective assistance in predicting the duration. However, further validation of the robustness and generalizability of the model is needed through prospective and larger datasets. Fourth, predicting the clinical outcomes of CA requires the unification of the CA procedures, which can only be adequately validated in prospective studies. Due to the limitations of the retrospective data in this study, we determined that it was not appropriate to predict the clinical outcomes, but in the future, we will evaluate the predictions regarding clinical outcomes using prospective data with a uniform CA technique. Lastly, the effect of rate control drugs as a confounding factor may have affected the HR. However, in this study, we dared to use data reflecting the actual clinical environment to improve the applicability of the model. In clinical practice, rate control with drugs is common, and by training the data to reflect that situation, we built a more practical model.

An ML model was developed to diagnose the duration of AF. The diagnostic accuracy of the cardiologist was improved with the support of the ML model. However, cardiologists did not entirely rely on the prediction of the ML model, even if they were aware of the limitations of their diagnostic capability.