Research in genetic counseling has increased with advances in diagnostic testing and treatment of genetic diseases [1]. Genetic counseling requires highly specialized skills, such as effectively communicating complex, evidence-based medical information in a clear and accessible manner, and providing essential mental health support. Despite rising demand, there remains a shortage of qualified professionals in this field [2]. In Japan, students can become certified genetic counselors by completing a graduate course at a graduate school with an accredited training program for genetic counselors. However, as of December 2023, only 389 qualified genetic counselors were available, highlighting the challenge of meeting the demand for genetic counseling services [3].

In recent years, the rapid development of large language models (LLMs) has led to their widespread application across various fields. Notably, the ChatGPT and GPT-4 developed by OpenAI have demonstrated human-level performance in diverse professional examinations [4] and even succeeded in the Japanese National Medical Examination [5-7] and the General Medicine In-Training Examination [8]. LLMs tailored for the medical field, such as Google’s Med-PaLM2, have demonstrated the ability to provide responses preferred by patients over those of doctors [9,10]. In addition, Sukeda et al [11,12] conducted domain adaptation for the medical fields on several Japanese LLMs. However, there are no studies specifically examining Japanese LLMs’ medical proficiency in genetic counseling. It is crucial not only to measure the general medical capabilities of LLMs through medical examinations but also to have experts evaluate LLMs in specialized tasks within the medical field.

In genetic counseling, where handling personal information requires the utmost care, lightweight, high-performance LLMs capable of offline operation are essential. This is due to the sensitive nature of the information involved, including family history, genetic data, and future health risks, which necessitate stringent privacy protection for the entire family. Unlike general medical practices that primarily impact individual patients, genetic information has extensive implications for life planning, family planning, and future generations. For example, the discovery of a genetic mutation associated with breast cancer not only affects the patient but also requires comprehensive counseling for his or her entire family. Similarly, identifying hereditary disease risks involves assessing genetic risks for future children.

This study introduces the development of an LLM for genetic counseling in Japanese, termed the “Japanese genetic counseling large language model” (JGCLLM). Specifically, we aim to explore effective enhancement techniques for LLMs and assess the responses of JGCLLM through expert evaluation. This research represents the first comprehensive study to analyze the impact of various enhancement techniques for LLMs in Japanese genetic counseling, marking a significant contribution to the field. Furthermore, we plan to leverage evaluation data to further enhance LLM performance through techniques, such as reinforcement learning from human feedback (RLHF) [13], which uses human preferences to guide the model’s learning and direct preference optimization (DPO) [14], directly optimizing the model based on pairwise comparisons of the outputs.

We applied standard LLM enhancement techniques, including instruction tuning [15], retrieval-augmented generation (RAG) [16], and prompt engineering, to lightweight Japanese LLMs. These techniques provide targeted solutions to key challenges in genetic counseling by improving response accuracy and safety. Instruction tuning enables the model to learn the appropriate response formats used by genetic counselors and to manage general inquiries with greater precision. RAG allows the model to base answers on the latest medical knowledge by referencing up-to-date literature or offering insights from previous patient records. Finally, prompt engineering ensures that the model adheres to safety and content guidelines, fostering responses that are both accurate and aligned with best practices in the field. Together, these combined techniques enhance the overall reliability and safety of artificial intelligence (AI)–driven genetic counseling.

Medical dialogue references for these methods were sourced from the web and developed by experts. Furthermore, we collected 1000 questions on genetic counseling through crowdsourcing and carefully selected 120 questions for assessment of the JGCLLM. Two certified genetic counselors and 1 ophthalmologist (SK, YU, and AY) were tasked with evaluating the response of the JGCLLM to these questions. The JGCLLMs were domain adapted using various combinations of methods. This process allowed us to analyze the impacts and challenges of these methods in the genetic counseling context. Figure 1 provides an overview of the study’s experimental design. Figure 1A shows the workflow of LLM enhancement techniques and datasets used, while Figure 1B shows a JGCLLM response with professional evaluation results across 4 criteria. Since the experiments were conducted in Japanese, this paper presents their descriptions translated into English, with the original Japanese versions shown in Multimedia Appendix 1 for reference.

To develop a lightweight LLM capable of offline execution, we opted for a publicly available 7B model instead of using application programing interfaces, such as GPT-4. Our selection process focused on Japanese language performance and efficiency within the medical domain.

Our selection criteria encompassed 2 key elements: the ELYZA-tasks-100 benchmark results [19] and the tokenization efficiency of words in the Manbyo dictionary [20]. ELYZA-tasks-100 [21] is a meticulously created dataset of 100 diverse and complex Japanese language tasks designed to assess the comprehensive language capabilities of models, such as ChatGPT. We used human evaluation to measure AI performance accurately, addressing the limitations associated with automatic evaluation metrics. The evaluation process is detailed later in the “Professional Evaluation” section.

Using these criteria, we examined 6 publicly available 7B-sized LLMs. We analyzed the published results of the ELYZA-tasks-100 [19] for each model and evaluated their tokenization efficiency with the Manbyo dictionary, which provides a standard set of clinical disease names in Japan. The ELYZA-tasks-100 scores and average Manbyo dictionary token counts for all 6 candidate models are listed in Table 4.

Based on this comprehensive analysis of the 6 models, we identified calm2-7b-chat as our baseline LLM owing to its superior performance in both metrics among the 7B models. This approach enabled us to identify a well-suited model for Japanese medical applications.

Two certified genetic counselors and 1 ophthalmologist (SK, YU, and AY) assessed the responses generated by the LLM to the 120 questions based on 4 key criteria: inappropriateness of information, sufficiency of information, severity of harm, and alignment with medical consensus. These evaluation criteria were adapted from Google’s Med-PaLM study [9]. The details are shown in Textbox 5.

To evaluate the effectiveness of the 3 LLM enhancement techniques—instruction tuning, RAG, and prompt engineering—we conducted a comparative analysis using 4 specific model configurations. These configurations were chosen as the minimal set required to reduce the evaluator’s workload while capturing the necessary data for the analysis:

The effect of instruction tuning was assessed by comparing the IT model with the Baseline model. The influence of the RAG is evident in the difference between the IT+RAG and IT models. Finally, the contribution of prompt engineering was demonstrated by comparing the IT+RAG+EP and IT+RAG models.

This research was approved by Kobe City Medical Center General Hospital, after ethics approval, including the Nara Institute of Science and Technology (review ezn240501).

The evaluation results of the JGCLLM by the 2 certified genetic counselors and 1 ophthalmologist (SK, YU, and AY) are shown in Figure 2 comprising 120 questions with 4 types of responses, for a total of 480 responses divided among 3 persons. Figure 2A shows the inappropriateness of information, Figure 2B illustrates the sufficiency of information, Figure 2C highlights the severity of harm, and Figure 2D details the alignment with medical consensus. The specific increases or decreases in the numbers resulting from instruction tuning, RAG, and prompt engineering are listed in Table 5.

RAG demonstrated notable improvements, increasing appropriate responses in 8 cases and reducing both low- and high-importance inappropriate information. In contrast, instruction tuning exhibited a concerning trend with a 14-case decrease in appropriate responses, primarily shifting to low-importance inappropriate information. Prompt engineering yielded mixed results, slightly increasing appropriate responses and also increasing high-importance inappropriate information.

RAG demonstrated the strong performance, increasing sufficient responses by 7 cases and notably decreasing high-importance missing information. Prompt engineering showed a mixed outcome, with a slight increase in sufficient responses but a substantial rise in cases requiring additional information. Instruction tuning slightly worsened the results, with a minor decrease in sufficient responses and an increase in missing low-importance information.

RAG delivered the highest favorable outcome, increasing harmless responses and reducing both moderate and severe harm cases. Instruction tuning displayed a concerning trend with fewer harmless responses and an increase in moderate harm cases. Prompt engineering yielded mixed results, slightly increasing harmless responses but also showing an increase in severe harm cases.

The RAG outperformed the other methods, increasing consensus-aligned responses and decreasing those that were not aligned with the consensus. Instruction tuning demonstrated a negative trend, significantly reducing consensus-aligned responses and increasing those opposed to consensus. Prompt engineering showed mixed results, primarily increasing responses with no consensus and slightly decreasing both aligned and opposed responses.

The analysis of instruction tuning revealed several concerning trends. First, inappropriate information in both low and high importance areas increased. The need for essential information also rose, suggesting a decline in the adequacy of information provided. Cases of moderate or minor harm increased, while cases with no harm decreased, indicating a potential rise in harm severity. Finally, the alignment with medical consensus significantly decreased, with more information conflicting with consensus, suggesting a deviation from the accepted medical standards. General-purpose LLMs should avoid answering medical questions and refrain from providing direct medical advice, instead encouraging consultations with specialists [26]. Therefore, the use of QA data in the medical field has resulted in the generation of in-depth medical answers, which may have influenced the poor evaluation results. Also, fine-tuning LLMs on new knowledge not acquired during pretraining can potentially encourage the generation of unfounded information [27].

In contrast, the results for RAG were positive. Appropriate information increased and inappropriate information of both low and high importance decreased, indicating notable improvements. Moreover, the sufficiency of information increased, indicating that a more comprehensive provision of information required less supplementation. Furthermore, the severity of harm decreased with fewer instances of moderate, mild, or severe harm. The alignment with medical consensus also improved with a decrease in nonconsensus information and an increase in information aligned with consensus, demonstrating better adherence to the established medical guidelines. However, data used for RAG in this study were relatively limited, with only a few cases referencing information relevant to the questions posed. Therefore, the impact of these findings on performance improvement is expected to be significant, particularly in increasing the quantity and quality of diverse RAG data in the future.

The findings on prompt engineering presented a more nuanced perspective. While low-importance inappropriate information decreased, inappropriate information of high importance increased, highlighting a concerning trend. The sufficiency of information also showed mixed results, with a decrease in low-importance issues but an increase in high-importance issues, underscoring both progress and critical deficiencies. Furthermore, the severity of harm exhibited a mixed outcome, with a decrease in moderate or mild harm but an increase in severe harm. Finally, alignment with medical consensus showed an equal decrease in both aligned and opposing information, indicating no significant improvement or decline in consensus adherence. Overall, while prompt engineering showed potential, further refinement is needed to achieve optimal results.

Effectiveness in the field of genetic counseling varied according to the characteristics of each method: RAG showed the most promising results, with the potential to significantly improve the quality of genetic counseling by allowing real-time access to up-to-date genetic databases for accurate, personalized advice. In contrast, instruction tuning displayed several worrisome trends, indicating a need for cautious application. Prompt engineering showed mixed results, warranting further investigation. Based on these results, RAG currently appears to be the promising approach for enhancing the quality of genetic counseling. Meanwhile, instruction tuning and prompt engineering require further refinement and careful application. A hybrid approach that appropriately combines these methods could further improve the quality and efficiency of genetic counseling in the future.

Several examples of inappropriate responses generated by JGCLLM are listed in Table 6. These examples cover various sensitive topics, such as advocating prenatal testing, criticizing relatives, assigning liability, and making probabilistic statements. The generated responses highlight the potential risks and ethical considerations of using LLMs to provide medical guidance.

Promoting prenatal testing raises significant ethical concerns by potentially influencing parental decisions regarding childbirth, which could lead to selective decision-making, such as opting for termination if a congenital disease is detected. Criticizing relatives for issues such as alcoholism can create discomfort for patients, exacerbate family tensions, and increase psychological distress. Assigning liability to third parties, such as genetic counselors, is problematic because the AI’s response may unfairly allocate responsibility, potentially leading to confusion. Communicating probabilities, such as the likelihood of inheriting allergic conditions, can adversely affects a patient’s mental well-being and influence reproductive decisions, underscoring the need to communicate probabilities with care and sensitivity.

Regulating these inappropriate LLM-generated responses requires rule-based controls at the term level, as illustrated in the probability statement example in Table 6, and context-aware assessments supported by machine learning, as demonstrated in the examples of promoting prenatal testing, criticizing relatives, and assigning liability. Ensuring the medical accuracy and evaluating whether LLM-generated responses comply with ethical standards are imperative.

In this study, we applied LLM enhancement techniques, such as instruction tuning, RAG, and prompt engineering, to calm2-7b-chat, a lightweight Japanese LLM, to create an LLM for Japanese genetic counseling (JGCLLM). In collaboration with certified genetic counselors and an ophthalmologist (SK, YU, and AY), we constructed and evaluated a QA dataset, assessing JGCLLM based on information inappropriateness, information sufficiency, harm severity, and alignment with medical consensus.

Analysis of instruction tuning revealed concerning trends, such as an increase in inappropriate information and a decrease in sufficient information and alignment with medical consensus. This shift may be attributed to transitioning from avoiding medical questions to providing detailed responses, which can potentially result in inappropriate medical information. Conversely, RAG demonstrated positive trends, showing improvements in appropriateness, sufficiency, harm severity, and consensus alignment. However, the limited data available for RAG highlight the need for a broader and higher-quality RAG dataset in future work to further enhance performance. Prompt engineering showed mixed results, with improvements in some criteria and notable deficiencies in others, indicating a need for further refinement.

When implementing LLM applications in the medical field, it is crucial to recognize that LLM-generated responses may contain medically inappropriate expressions. Ensuring medical accuracy and addressing ethical considerations are essential when using LLMs to provide medical guidance.