The questions asked to LLMs consisted of a patient questionnaire and an expert questionnaire, each containing 15 questions and covering aspects such as the diagnosis, treatment, prognosis, and follow-up of breast cancer. The patient questionnaire was derived by distributing a questionnaire to patients with breast cancer to investigate their most important concerns (Textbox 1). The expert questionnaire was summarized by two experienced breast surgeons, based on the National Comprehensive Cancer Network (NCCN) Clinical Practice Guidelines in Oncology for breast cancer and the International Consensus Guidelines for advanced breast cancer (Textbox 2) [15,16]. On January 15, 2024, all questions were input into ChatGPT (version 4.0) and EB (version 4.0). Each input was independently entered using the “new chat” function and inputted twice to detect its repeatability. To optimize the responses of the LLMs, prompt engineering was applied with the same lead-in statement: “Now that you are a breast cancer specialist, please answer the following questions,” which was input into the LLMs along with each question. As ChatGPT was developed in the United States, we queried ChatGPT in both Chinese and English, denoted as ChatGPT-Chinese (ChatGPT-C) and ChatGPT-English (ChatGPT-E), respectively, and the EB, developed in China, was queried in Chinese only. The responses from ChatGPT and EB were recorded using Microsoft Excel (Multimedia Appendix 1).

The breast cancer multidisciplinary treatment team scored the accuracy, practicality, and generalization-specificity of each response using a Likert scale, with the poorer of the two responses being included in the final score if the responses were inconsistent. The team consisted of 13 members, including 7 breast cancer specialists, 2 imaging specialists, 2 pathology specialists, and 2 nursing specialists. The Likert scale is a hierarchical scale, originally developed by Likert and has been used extensively in several research studies [17,18]. Accuracy was divided into 6 levels from 1 to 6, with higher scores indicating better accuracy (Table 1). Practicality was divided into 4 levels from 1 to 4, with higher scores indicating better practicality (Table 2). The generalization-specificity score (GSS) is divided into 5 levels from 1 to 5, with higher scores indicating better specificity (Table 3). To reduce bias caused by individual differences in understanding the scoring system, all experts reviewed and discussed the scoring criteria of the Likert scale before the assessment.

The Shapiro-Wilk test was used to determine the normality of the scores and the Levene test was used to evaluate the homogeneity of variance. Differences between two groups were assessed using the Wilcoxon rank sum test. The Kruskal-Wallis test evaluated differences between three or more groups of variables, and the Dunn test was used for two-way between-group comparisons of variables that were not normally distributed. P<.05 was deemed statistically significant. The intraclass correlation coefficient (ICC) was used to evaluate the consistency of accuracy, practicality scores and GSS among 13 raters. An ICC ≥0.75 was considered to indicate good consistency. All statistical analyses were performed using R software (version 4.0.3; R Foundation for Statistical Computing).

This study did not gather patient data and did not involve human subjects. Therefore, approval by the institutional review board of Chongqing Medical University was not required.

We have reported several important findings in this study. First, based on the patient questionnaire responses, ChatGPT-E demonstrated significantly higher accuracy compared to ChatGPT-C and EB in addressing questions related to breast cancer surgery treatment (Q4) and postoperative management (Q6, Q11, and Q15) (Figure 2A). Additionally, ChatGPT-E’s responses to questions concerning breast cancer staging (Q2), treatment (Q4 and Q8), and postoperative management (Q6 and 14) were more comprehensive and practical (Figure 2). In the expert questionnaire, ChatGPT-E demonstrated similar advantages, especially for breast cancer drug treatment (Q8) and follow-up (Q15), with more comprehensive, accurate, and practical responses, reflecting higher efficiency (Figure 3). Overall, ChatGPT-E performed the best in both patient and expert questionnaires. Despite the advantages in training strategies that may have enabled ChatGPT-C to perform better than EB in answering general questions from patients with breast cancer, the performances of both models were unsatisfactory while answering comparatively specialized questions in the field of breast cancer in the Chinese-language context (Figure 1). For example, in response to the expert questionnaire Q5, both ChatGPT-C and EB only briefly mentioned several common methods of first-stage breast reconstruction. Only ChatGPT-E mentioned “latissimus dorsi flap breast reconstruction” and briefly introduced the advantages of each surgical method. Although its response was not detailed enough, the basic framework was correct. Although the overall hallucination rate of ChatGPT-E was significantly lower than that of ChatGPT-C and EB (both P<.001), 11.79% (23/195) of its responses in the expert questionnaire still contained inaccurate information. This finding indicates that even models with relatively superior performance must further reduce hallucination rates in the specialized field of breast cancer to meet clinical requirements.

In addition, we found that ChatGPT-C’s responses to the 15 questions in the patient and expert questionnaire each contained one notable medical terminology translation error. For instance, “泛素酮 (Tamoxifen)” and “莱特罗唑 (Letrozole)” were mentioned in the response to expert questionnaire Q7. In patient questionnaire Q12, the term “port” in “subcutaneously implanted infusion port” was translated as “harbor” (ie, 港口 in Chinese). Patients may misinterpret postoperative care requirements due to this nonstandard translation, potentially leading to complications. In the medical domain, English is the primary language for international academic communication. ChatGPT’s core training data is predominantly in English, including extensive English medical literature and clinical guidelines (eg, National Comprehensive Cancer Network and Advanced Breast Cancer). Compared with EB, ChatGPT has greater access to and understanding of these professional resources, enabling it to provide more comprehensive and accurate information when answering related questions. The inferior performance of ChatGPT-C compared to ChatGPT-E may stem from limitations in cross-language processing. Although ChatGPT supports multiple languages, the semantic structure and medical terminology in Chinese differ significantly from English. During cross-language processing, ChatGPT may rely on translation mechanisms rather than native Chinese training, leading to semantic distortion in specialized content and reduced answer quality. Preliminary testing in other languages such as Spanish and French has identified similar issues when dealing with proper nouns (eg, medications, surgical procedures) and compound terms. The model tends to rely on literal translation or the creation of neologisms rather than following localized standards, which may lead to ambiguities. However, a recent study by Tian et al [19] on CHIMED-GPT found that pretraining GPT using a specific Chinese medical dataset made CHIMED-GPT perform significantly better than other models in tasks such as multiple choice and open-ended responses. To address the translation errors in ChatGPT-C’s responses to breast cancer–related questions, fine-tuning the model with Chinese medical datasets represents an effective improvement strategy. These datasets should include a wide range of Chinese medical literature, clinical guidelines, case reports, and patient-doctor dialogues specific to the field of breast cancer. By training the model on these specialized datasets would help it grasp the nuances and context of medical language better, leading to more accurate translations and responses. In addition, user error correction interfaces can be designed to allow physicians or patients to flag translation errors (eg, “莱特罗唑 → 来曲唑"), and the system could then automatically collect these error cases and add them into fine-tuning datasets, thus achieving continuous model optimization. In terms of response repeatability, the performance of ChatGPT-E (52/60, 86.67%) was significantly better than ChatGPT-C (50/60, 83.33%) and EB (40/60, 66.67%).

It is worth noting that we have also found a lack of corresponding empirical data and references to support the views of the two LLMs in their responses, which could undermine the credibility and practicality of their responses, especially in evidence-based clinical practice [20,21]. For example, in responses to the expert questionnaire Q14 and patient questionnaire Q7, both LLMs mentioned that “chemotherapy affects fertility or breastfeeding,” but failed to provide any useful references, resulting in compromised credibility and possibly inability to guide physicians and patients in making correct decisions. In addition, the responses of LLMs were relatively generalized, indicating that they were widely mentioned but lacked specificity, similar to the findings of the study by Giannakopoulos et al [22] who used LLMs to answer dental-related questions. This generalized responses also resulted in the LLMs being less accurate and practical in answering the expert questionnaire than the patient questionnaire (Figure 1). For example, in the responses to expert questionnaire Q15 and patient questionnaire Q14, although LLMs mentioned the need for regular follow-up and corresponding examinations for patients with breast cancer, they did not provide specific answers. These generalized responses are of limited value to clinical professionals, who require highly accurate, comprehensive, and professional information, similar to previous studies on ChatGPT’s responses to mental health and liver cancer–related questions [23,24]. However, they may be beneficial for patients with breast cancer who lack medical expertise, as the responses from LLMs already covered the vast majority of the questions and were comparable to clinician responses, similar to the findings from a study by Endo et al [25] on the use of LLMs for answering questions related to liver transplantation. Generalized LLM–generated information poses risks ranging from clinical mismanagement to ethical violations, particularly in complex fields such as breast cancer. Given that in breast cancer treatment, timely decision-making is critical early diagnosis and intervention significantly improve cure and survival rates. Vague recommendations from LLMs may put patients at risk of missing the optimal therapeutic window, thus potentially exacerbating disease progression. While LLMs such as ChatGPT-E show promise in patient education, their utility depends on the patients’ ability to contextualize and validate the outputs provided. Patients should maintain a cautious attitude toward responses generated by LLMs that lack personalized recommendations and refrain from relying on them exclusively. It is recommended that patients use the information provided by LLMs as a general reference, while promptly communicating with professional physicians. By integrating their specific clinical circumstances, patients can obtain accurate and personalized medical advice and guidance to safeguard their health and safety.

Furthermore, LLMs have exposed potential ethical risks in responding to breast cancer–related questions. While clinical trials may offer access to the latest therapeutic regimens and advanced technologies, they inherently carry uncertainties and potential adverse effects. When responding to Patient Questionnaire Q2 and Expert Questionnaire Q6, LLMs encouraged patient participation in clinical trials without adequately explaining the risks and uncertainties involved. This could lead patients to assume unnecessary risks without being fully informed [26]. We recommend establishing a dedicated review team to systematically audit medical recommendations provided by LLMs, particularly regarding clinical trial recommendations and vague suggestions. This ensures that LLMs responses adhere to medical ethical standards and professional norms, and correct or block responses that do not meet the requirements. At the same time, the issue of data security involved in LLMs is becoming increasingly prominent [27]. Although the responses of the LLMs in this study did not inadvertently leak sensitive information and were based on general medical knowledge and standardized recommendations, some studies have shown that LLMs may inadvertently memorize and disclose original data in their responses [28,29]. In a study by Nasr et al [30], researchers were able to extract training data for various LLMs including ChatGPT through specific “attacks”. Therefore, doctors or health care organizations should obtain informed consent from patients when using real patient data for model training and application to LLMs, while ensuring the anonymization and deidentification of data [26]. Patients should also be trained in data security awareness and instructed to avoid disclosing private personal information while using LLMs to obtain health information [31]. China has established a series of laws and regulations, including the Personal Information Protection Law (PIPL) to regulate data processing and applications. However, specific regulatory details for LLM–based medical applications are still being explored and refined. The effective implementation of these laws and regulations in the LLM–based medical applications is currently a concern [32]. Based on the results of this study, future regulation should focus on standardizing the LLM–based medical applications. It is essential to strictly regulate data processing procedures to ensure data security and privacy protection in compliance with laws and regulations such as the PIPL. Further, an ethical review mechanism for LLM–based medical applications should be established, clarifying ethical guidelines in aspects such as medical information provision and patient decision-making guidance to prevent ethical risks [33]. Additionally, explicit limitations should be imposed on the scope and modalities of PIPLs in health care to prevent their excessive involvement in core medical operations when sufficient reliability is not assured.

This study reflects some important issues that may arise when using LLMs in clinical scenarios related to breast cancer in China. Overall, LLMs can serve as effective tools for Chinese patients with breast cancer to obtain health information, helping to address the majority of concerns related to diagnosis, treatment, recovery, and follow up of this population. However, in the context of breast cancer specialists, the accuracy, practicality, and relevance of LLMs’ responses need improvement. We propose a multidimensional optimization framework to enhance the utility and reliability of ChatGPT in breast cancer diagnosis and management. On the one hand, the model should be trained using high-quality medical data, such as the latest breast cancer research, clinical guidelines, and case reports, to improve its accuracy and practicality in the professional domain. On the other hand, under ethical compliance, ChatGPT should be connected to deidentified electronic health records, laboratory systems, and imaging databases to access real-time patient data and provide more personalized recommendations. Based on our research, ChatGPT-E demonstrates better repeatability, accuracy, and practicality in its responses compared to other LLMs. Therefore, it is recommended that Chinese patients with breast cancer translate their questions into English before querying ChatGPT, to improve its effectiveness. In addition, considering the potential data security, ethical, and legal risks of LLMs in clinical practice, it is essential to strengthen regulation of the training and application of LLMs in the medical professional field [34]. This study has certain limitations as the response from LLMs were not applied in real time to address the questions of patients with breast cancer or to assist doctors in making clinical decisions. We also did not evaluate all issues related to breast cancer. The data collection for this study was completed in January 2024. However, certain models (eg, GPT-4-turbo and DeepSeek) had not been publicly released at that time or failed to provide stable interfaces for academic research applications and therefore were not included in this study. Additionally, this study used structured questionnaires to evaluate the responses of LLMs, which ensured standardized assessment but partially limited the assessment of LLMs’ ability to handle open-ended, unstructured, and interactive questions. Future iterations could incorporate open-ended or interactive question types to better simulate real-world clinical consultations. Lastly, patient and expert user feedback can provide critical user-perspective data, address the limitations of existing expert-only evaluations, and enhance the application effectiveness and user experience of LLMs in health care. Further research is required to evaluate the real-world clinical effectiveness of LLMs and the real user experience of patients with breast cancer in China.