Leveraging Retrieval-Augmented Large Language Models for Dietary Recommendations With Traditional Chinese Medicine’s Medicine Food Homology: Algorithm Development and Validation

Introduction

Background

The concept of “medicine food homology” (MFH) in Traditional Chinese Medicine (TCM) means that certain substances can function as both nutritional food and therapeutic medicines depending on their application and dosage [1]. This ancient philosophy has gained considerable traction in modern health care systems [2], offering approaches to disease prevention and management through dietary interventions [3]. However, the practical implementation of MFH principles remains largely dependent on the expert knowledge of TCM practitioners, leading to inconsistent application and limited accessibility. The absence of automated and systematic approaches to using MFH principles for dietary recommendation generation has hindered its broader integration into contemporary health care frameworks and constrained its potential benefits for public health.

Recent advances in large language models (LLMs) [4] have demonstrated their remarkable performance across a variety of medical tasks [5], including automated clinical decision support [6], medical education [7], and health care information retrieval [8]. These capabilities are primarily derived from extensive pretraining on large-scale corpora, which encode rich knowledge from diverse textual sources. The application of MFH principles heavily depends on the domain knowledge documented in TCM literature, including foods, herbs, and formulations. Given that many LLMs have been pretrained on the corpora, which incorporate such TCM literature, combined with their strong natural language generation abilities, they present promising potential for automating MFH-based dietary recommendations. Nonetheless, LLMs exhibit substantial limitations as they often produce hallucinations that refer to factually incorrect information [9]. This problem is especially evident when generating information related to the medicinal properties of food in the context of TCM. For example, when querying with “I have stomach discomfort recently, can you recommend some summer drinks?” the LLMs respond: “Mung bean soup clears heat and is suitable for heat-related discomfort.” However, according to MFH principles, mung beans are cold (yin), and their consumption may aggravate stomach discomfort. Such inaccuracies significantly affect the safety and reliability of LLMs in practical health care applications [10].

Retrieval-augmented generation (RAG) has emerged as a promising strategy to mitigate these limitations by incorporating external knowledge into LLMs, thereby enhancing factual accuracy and reducing hallucinations [11]. However, using TCM knowledge as the retrieval source of RAG presents unique challenges. TCM knowledge is sourced from diverse and heterogeneous materials, including ancient and classical literature, modern TCM literature, food and herbs information, regulatory documents, research papers, and TCM formulations. This multisource nature results in inconsistent knowledge representation, terminological ambiguities, and varying levels of evidence quality. Furthermore, MFH principles inherently involve uncertainties arising from dosage effects, individual variability, and context-dependent applications [12]. Currently, there is no comprehensive and structured knowledge base specifically tailored for supporting RAG grounded in MFH principles, creating a significant gap between traditional wisdom and modern technologies.

Related Work

Dietary Recommendation Systems

Dietary recommendation systems provide personalized food and nutrition suggestions based on individual health status and preferences. Traditional approaches have primarily relied on rule-based methods using expert knowledge and predefined guidelines, often lacking personalization and accessibility. Recent advancements in artificial intelligence have transformed this field through deep learning and knowledge graphs (KGs). Chen et al [13] defined food recommendation as constrained question answering over a food KG, integrating user preferences with health guidelines. PPKG [14] is a cancer-specific dietary recommendation system that combines a KG with time-aware long short-term memory networks to dynamically adjust recommendations for cancer prevention and rehabilitation. Ma et al [15] proposed a nutrition-related KG method based on graph convolutional networks to enhance food recommendation diversity while promoting more healthy eating habits. LLMs have further advanced dietary recommendation capabilities. ChatDiet [16] integrates personal and population models with causal inference to enhance personalization and explainability. Jin et al [17] demonstrated the clinical effectiveness of a GPT-based dietary recommendation system for patients with hemodialysis, successfully reducing serum potassium levels. Kopitar et al [18] created a generative artificial intelligence system for personalized inpatient meal planning that incorporates electronic health records and clinical guidelines.

However, we still do not have methods to integrate MFH principles with LLMs for dietary recommendation. Our framework bridges this gap by combining MFH KG with LLMs to develop more comprehensive and culturally informed dietary recommendations.

The Interplay Between LLMs and KGs

KGs serve as structured representations of factual knowledge, providing an excellent complement to parameterized language models [19,20]. With the emergence of LLMs, research has evolved along 2 complementary trajectories: leveraging LLMs to construct KGs and enhancing LLMs with KG integration. Several approaches have emerged for LLM-assisted KG construction. Extract-define-canonicalize [21] addresses scalability through open information extraction (OpenIE) and post hoc canonicalization. BEAR [22] leverages LLMs for zero-shot knowledge extraction in service-oriented domains. LLM-TIKG [23] applies few-shot learning for constructing threat intelligence KGs, while Yang et al [24] presented an approach for medical ontology expansion using LLMs to generate competency questions.

Researchers have also explored integrating KGs to enhance LLMs’ capabilities. GraphGPT [25] combines LLMs with graph structural knowledge through instruction tuning. GaLM [26] transforms KGs into text representations to improve reasoning while reducing hallucinations. Think-on-Graph [27] is a training-free framework that uses iterative beam search to enable interactive reasoning, while StructGPT [28] enhances zero-shot reasoning through an iterative Reading-then-Reasoning approach. Our work builds upon these dual research trajectories by proposing a framework that leverages LLMs for KG construction and uses KGs to enhance the capabilities of LLMs specifically within the domain of TCM, thereby providing reliable dietary recommendations grounded in MFH principles.

Objectives

To overcome the limitations of existing methods, we propose a new framework that enhances LLMs’ capability to generate MFH dietary recommendations by using an uncertain knowledge graph (UKG) as an external resource. This framework uses multistep LLM calls for OpenIE [29] to automatically construct an MFH KG from unstructured and heterogeneous literature sources. It then implements UKG reasoning to measure the confidence of existing triples and complete missing triples. To improve knowledge retrieval, the framework adopts a reasoning path–based knowledge retrieval strategy, extracting relevant knowledge from the MFH KG and refining the results through ranking and filtering based on triple confidence scores and entity importance. By encoding domain-specific knowledge and addressing uncertainties related to MFH, the MFH KG empowers LLMs to deliver more personalized, evidence-based, and reliable dietary recommendations.

The key contributions of this study are as follows: (1) an integrated framework that leverages KG-augmented LLMs to generate personalized and evidence-based MFH dietary recommendations, (2) an LLM-driven OpenIE methodology for automatically constructing a UKG specifically tailored to TCM’s MFH from heterogeneous multisource data, (3) a systematic evaluation of several mainstream LLMs for the generation of MFH dietary recommendations, and (4) experimental results and case studies that demonstrate the capability of the proposed framework to deliver personalized and professional MFH-based dietary recommendations.

Methods

Overview

This study proposed Yaoshi-RAG, an RAG framework designed to enhance LLMs’ capabilities in generating evidence-based dietary recommendations based on MFH principles. As illustrated in Figure 1, the framework consists of 2 modules:

1. Knowledge graph construction: A comprehensive corpus was compiled from multiple sources, and an MFH KG was constructed using LLM-driven OpenIE. Then, UKG reasoning was applied to measure the confidence of extracted triples and complete missing triples.
2. Retrieval-augmented generation: Given a user query, the framework first identified and linked query entities to the MFH KG. The LLM then constructed relation paths to facilitate the retrieval of reasoning paths. Subsequently, these candidate reasoning paths were refined through postprocessing, including ranking and filtering, to extract the top-$k$ most relevant reasoning paths, thereby improving retrieval accuracy. The retrieved knowledge was integrated with the user query through prompt engineering, guiding the LLM to generate personalized dietary recommendations that aligned with both the user’s needs and MFH principles.

‎

A comprehensive corpus was compiled from multiple sources, and an MFH KG was constructed using LLM-driven OpenIE. Then, UKG reasoning was applied to measure the confidence of extracted triples and complete missing triples. In the following sections, we provide a detailed explanation of both modules, outlining their design, implementation, and contributions to the overall framework.

KG Construction

Corpus Collection

To construct an MFH KG, we systematically constructed a corpus from diverse sources, including ancient and classical literature, modern TCM literature, food and herbs information, regulatory documents, research papers, and TCM formulations. This process results in a corpus of 1359 relevant documents. Table 1 shows the statistical distribution of documents across different source categories.

To improve the efficiency and accuracy of entity-relation triple extraction, each document was segmented into passages of 300 tokens. These passages were then input into LLMs with a carefully designed prompt (Textbox S1 in Multimedia Appendix 1) to resolve pronominal references by incorporating information from document titles and neighboring passages. Additionally, irrelevant content within passages was filtered out to ensure domain-specific focus.

MFH Knowledge Graph

Based on the collected corpus, we constructed an MFH KG $G$, as shown in Figure 2, which illustrates the detailed process. Formally, the graph is defined as a triple $G=(E,R,F)$, where $E$ represents the set of entities, $R$ represents the set of relations, and $F\subseteq E\times R\times E$ represents the fact set. Each fact in $F$ is represented as a quadruple $(h,r,t,s)$, where $h$ is the head entity, $r$ is the relation, $t$ is the tail entity, and $s$ is the confidence score reflecting the likelihood that the triple $(h,r,t)$ holds true. For instance, consider the quadruple ($h$: Insomnia and Vivid Dreams, $r$: Related Constitution, $t$: Yin-Deficient Constitution, $s$: 0.87). This quadruple indicates that Insomnia and Vivid Dreams are associated with the Yin-Deficient Constitution, with a confidence score of 0.87.

‎

We used LLM-driven OpenIE to extract triples $\left(h,r,t\right)$ from each passage within the corpus. Specifically, we first integrated predefined relation definitions into a prompt (Textbox S2 in Multimedia Appendix 1), guiding the LLM to identify and extract potential relations. These candidate relations, along with existing ones, were subsequently processed using another prompt (Textbox S3 in Multimedia Appendix 1), where the LLM determined whether to retain new relations or filter out redundant ones. This process resulted in the construction of a relation set ${\displaystyle r_{\mathrm{s}\mathrm{e}\mathrm{l}}}$, which was incorporated into OpenIE prompts, instructing the LLM to extract entity-relation triples. The relations of extracted triples were strictly constrained to the relation set ${\displaystyle r_{\mathrm{s}\mathrm{e}\mathrm{l}}}$.

Specifically, during the OpenIE, we instructed the LLM to extract attributes of food entities and dish entities (Textboxes S4 and S5 in Multimedia Appendix 1). For food entities, LLMs referenced their source passages to update attributes, including name, primary effects, consumption methods, indications, and contraindications. Similarly, for dish entities, attributes such as name, ingredient composition, cooking method, and dietary benefits were updated.

The automated construction of the MFH KG inevitably introduced noise, incompleteness, and factual inaccuracies, making absolute factual correctness unattainable. To mitigate these issues, we adopted UKG reasoning to measure the confidence of extracted triples and complete missing triples. Specifically, we first used unKR [30], an open-source Python library for UKG reasoning, to learn vector representations of entities and relations. Based on these representations, we applied the triple confidence measurement method [31], integrating heterogeneous evidences to measure confidence of each triple. For each given head entity $h$ and relation $r$, candidate tail entities were enumerated to construct new triples absent from the current graph. The triples with confidence scores larger than 0.85 were added to the MFH KG.

Retrieval-Augmented Generation

The Retrieval Strategy

Given a user query ${\displaystyle \mathrm{Q}}$, we first constructed a prompt (Textbox 1) to guide the LLM to extract a set of relevant keywords ${\displaystyle \mathrm{E}}$ from ${\displaystyle \mathrm{Q}}$. We then encoded both the keywords and all entities from the MFH KG into dense vector representations, denoted as $H_R$ and $H_G$, respectively. Considering that SBERT [32] has demonstrated superior performance over BERT [33] and RoBERTa [34] on semantic textual similarity tasks while maintaining lower computational overhead [35], we adopted SBERT for embedding both the extracted keywords and entities. Subsequently, we computed the cosine similarities between the embeddings of each keyword and all entities. For each keyword, we retrieved the top-10 most similar entities from the MFH KG based on cosine similarity scores. These candidate entities were then presented to the LLM, which was instructed to identify the matched entity (Textbox 2). The selected entity corresponding to each keyword constitutes the query entity set ${\displaystyle E_{\mathrm{Q}}=\left\{e_1,e_2,\dots \hspace{0.17em},e_n\right\}}$, which serves as the starting point for subsequent reasoning path retrieval.

This study used path-based retrieval from the MFH KG rather than retrieving contextual subgraphs for linked entities. This is motivated by the advantages reasoning paths offer in generating MFH-based dietary recommendations. Specifically, reasoning paths provide structured knowledge by capturing direct relations between query entities and answer entities. For example, the reasoning path “(Insomnia and Vivid Dreams, Related Constitution, Qi-deficient Constitution); (Qi-deficient Constitution, Suitable Food, Codonopsis)” enables LLMs to clearly infer the relations between a user’s symptoms, their constitution, and the recommended food, thereby enhancing the interpretability. In contrast, subgraph-based retrieval in long-hop reasoning tasks often introduces irrelevant information [36]. This not only leads to significant computational overhead but also hinders LLMs from effectively extracting relevant information from the retrieval results.

Built upon the previous work [37], we first provided the LLM with a structured prompt (Textbox 3) listing all relations in $G$. The LLM then generated multiple relation paths ${\displaystyle \left\{p_{\mathrm{r}\mathrm{e}\mathrm{l}}^1,p_{\mathrm{r}\mathrm{e}\mathrm{l}}^2,\dots ,p_{\mathrm{r}\mathrm{e}\mathrm{l}}^k\right\}}$, where each ${\displaystyle p_{\mathrm{r}\mathrm{e}\mathrm{l}}^i}$ is an ordered sequence of relations$\left\{r_1,r_2,\dots ,r_l\right\}$. These relation paths serve as guides for retrieving knowledge essential to answering user queries. Given that the MFH KG contained considerably fewer relations compared with large-scale KGs such as FreeBase [38], we did not fine-tune the LLM for relation path generation. Instead, we used few-shot prompting to enhance the LLM’s ability to generate meaningful relation paths.

Subsequently, starting from each entity in the ${\displaystyle {\mathrm{E}}_{\mathrm{Q}}}$, we followed each generated relation path ${\displaystyle p_{\mathrm{r}\mathrm{e}\mathrm{l}}^i}$ to retrieve a set of reasoning paths ${\displaystyle P_{\mathrm{r}\mathrm{e}\mathrm{a}}=\left\{p_{\mathrm{r}\mathrm{e}\mathrm{a}}^1,p_{\mathrm{r}\mathrm{e}\mathrm{a}}^2,\dots ,p_{\mathrm{r}\mathrm{e}\mathrm{a}}^m\right\}}$ from $G$, where each reasoning path ${\displaystyle p_{\mathrm{r}\mathrm{e}\mathrm{a}}^i=\left\{\left(e_1,r_1,e_2\right),\left(e_2,r_2,e_3\right),\dots ,\left(e_l,r_l,e_{l+1}\right)\right\}}$ consists of multiple triples $\left(e_j,r_j,e_{j+1}\right)$ and represents an instance of one of the relation paths.

Retrieval Result Postprocessing

In our initial experiment, we noticed that the reasoning paths in ${\displaystyle P_{\mathrm{r}\mathrm{e}\mathrm{a}}}$ contained redundant and irrelevant information. To enhance the quality of these reasoning paths, we computed a score for each reasoning path in ${\displaystyle P_{\mathrm{r}\mathrm{e}\mathrm{a}}}$ by considering both the entity importance and the triple confidence within the reasoning path. These scores enabled us to filter out less relevant reasoning paths.

The importance of each entity in the reasoning path was calculated using a weighted PageRank [39] algorithm. Traditional PageRank ignored the confidence of triples, which could lead to equivalent treatment of false and true relations, thus causing inaccuracies in entity importance. To overcome this limitation, we constructed a subgraph using entities from the ${\displaystyle P_{\mathrm{r}\mathrm{e}\mathrm{a}}}$ along with their one-hop neighbors and incorporated confidence scores of triples as edge weights in the PageRank computation. The weighted PageRank formula is defined as:

${\displaystyle {\displaystyle \begin{array}{c}\mathrm{P}\mathrm{R}\left(e_i\right)=\left(1-d\right)\cdot \frac{1}{N}+d\cdot \sum _{e_j\subset In\left(e_i\right)}\frac{w_{j\to i}}{\sum _k{w}_{j\to k}}\cdot \mathrm{P}\mathrm{R}\left(e_j\right)\end{array}}}$(1)

where ${\displaystyle \mathrm{P}\mathrm{R}\left(e_i\right)}$ represents the weighted PageRank score of entity $e_i$, $d$ is the damping factor, and $N$ is the total number of entities in the graph. The term $w_{j\to i}$ denotes the edge weight from entity $e_i$ to $e_j$, derived from the confidence score of the corresponding triples. When multiple relations exist between $e_i$ and $e_j$ with different confidence scores, their influence was aggregated by averaging these scores. For scoring the reasoning paths, we integrated entity importance and triple confidence scores to formulate the following path-scoring function:

${\displaystyle {\displaystyle \begin{array}{c}\mathrm{P}\mathrm{a}\mathrm{t}\mathrm{h}\mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}\left(P\right)=\prod _{i=1}^n{w}_{i-1\to i}\times \left(\frac{1}{n}\sum _{e_j\in P}\mathrm{P}\mathrm{R}\left(e_j\right)\right)\end{array}}}$(2)

where $w_{i-1\to i}$ represents the confidence score of the relation from entity $e_i$ to $e_j$, $n$ denotes the total number of entities in the path, and ${\displaystyle \mathrm{P}\mathrm{R}\left(e_j\right)}$ is the weighted PageRank score of entity $e_j$ within the reasoning path. This scoring methodology captured both the confidence of triples within the reasoning path and the importance of the entities involved, striking a balance between path reliability and informational importance. Finally, we ranked the reasoning paths based on their scores and retained only the top-$k$ reasoning paths for subsequent inference.

Answer Generation

We used prompt engineering to improve the accuracy and relevance of generated answers of LLM. Prompt engineering involves constructing carefully designed prompts to guide LLMs toward generating high-quality and task-specific responses. Specifically, we constructed a prompt template (Textbox 4) comprising four key components: (1) task specification, (2) user query, (3) retrieved reasoning paths, and (4) attributes of food and dish entities. This structured prompt facilitates the integration of external knowledge, improves the LLMs’ comprehension of MFH principles, and supports the generation of more accurate responses.

LLM Selection

To identify the most suitable LLM for MFH dietary recommendation tasks, we evaluated several mainstream LLMs, including GPT-4 [40], LLaMA2-Chat-7B [41], Qwen2.5-7B [42], and DeepSeek-R1 [43]. Additionally, we considered the influence of model size, incorporating both large-scale models (GPT-4 and DeepSeek-R1) and 7B-parameter models (LLaMA2-Chat-7B and Qwen2.5-7B). During the experiment, we implemented the same prompt set across all models to ensure comparative evaluation under identical input conditions. All models were integrated with our proposed framework for dietary recommendation generation based on the MFH principles.

Evaluation Metrics

Automated Evaluation

To quantitatively assess the effectiveness and accuracy of the model’s dietary recommendations, we used 2 evaluation metrics: $\mathrm{H}\mathrm{i}\mathrm{t}\mathrm{s}@1$ and F₁-score. $\mathrm{H}\mathrm{i}\mathrm{t}\mathrm{s}@1$ measures the proportion of queries where the top-ranked predicted recommendation matches a ground truth answer. This metric evaluates the model’s ability to prioritize the most relevant dietary recommendation. F₁-score captures the harmonic mean of precision (P) and recall (R), offering a balanced evaluation of both aspects. This metric is particularly appropriate in scenarios where multiple valid recommendations may exist for a single query. It is defined as follows:

${\displaystyle \begin{array}{c}{F}_1=2\times \frac{P\times R}{P+R}\end{array}}$(3)

where precision and recall are defined as:

${\displaystyle \begin{array}{c}P=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}},R=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}\end{array}}$(4)

In this context, ${\displaystyle \mathrm{T}\mathrm{P}}$ (true positives) represents the number of dietary recommendations correctly suggested, ${\displaystyle \mathrm{F}\mathrm{P}}$ (false positives) indicates inappropriate dietary recommendations incorrectly suggested, and ${\displaystyle \mathrm{F}\mathrm{N}}$ (false negatives) refers to appropriate dietary recommendations that the model failed to suggest.

Human Evaluation

While automated evaluation methods are widely applied in natural language processing tasks, they have certain limitations in the context of dietary recommendations. Automated metrics struggle to assess the effectiveness of recommendations, specifically whether the suggested food and their combinations align with MFH principles. Moreover, user acceptability and explainability of the generated dietary recommendations are difficult to accurately assess using automated methods. Therefore, we incorporated human evaluation, integrating subjective ratings from both experts and general users to provide a more comprehensive assessment of the generated dietary recommendations.

Specifically, we assessed the generated dietary plans based on the following five evaluation criteria: (1) rationality, (2) explainability, (3) user acceptability, (4) personalization, and (5) consistency. Rationality assesses whether the recommended dietary plan adhered to both TCM principles and modern nutritional standards. Explainability evaluates whether the recommendations included clear TCM-based justifications. A higher score indicates that the system provided reasonable and understandable explanations rather than simply outputting ingredient combinations. User acceptability measures how well the recommendations aligned with users’ subjective preferences, including taste, cultural adaptability, and feasibility. Personalization assesses whether the system provided tailored recommendations based on an individual’s constitution and health conditions. Consistency examines whether the model produced stable recommendations under similar inputs.

Among these evaluation criteria, rationality, explainability, user acceptability, and personalization were assessed using a 5-point Likert scale (1‐5, where 1 represented the lowest score and 5 represented the highest). Consistency was evaluated using a binary judgment (Yes=5 points and No=0 points).

To conduct the evaluation, we recruited 5 experts with backgrounds in TCM or nutrition, along with 20 general users. Participants followed a structured human evaluation guideline to rate the generated dietary recommendations. The evaluation was conducted using a survey system, where assessors were provided with the user queries along with the corresponding dietary recommendations generated by LLMs. Finally, we computed the mean scores for each evaluation criterion.

Ethical Considerations

Institutional review board approval was not required for this study, as it did not involve direct research with human participants. All corpora used, such as research papers and TCM formulations, were publicly available [44-46], and no identifiable private or sensitive information was accessed or recorded. Annotators involved in the human evaluation participated voluntarily, provided informed consent prior to participation, and were informed of their right to withdraw at any time. No compensation was provided, and no private information was collected.

Results

We constructed an MFH KG comprising 24,984 entities, 22 relations, and 29,292 triples. To evaluate the LLMs’ performance in dietary recommendation tasks, we developed a specialized question answering dataset consisting of 2000 queries focused on MFH principles. This dataset was designed to ensure balanced coverage across diverse health conditions and TCM constitutional types.

Table 2 shows a comparative analysis of LLMs with or with no augmentation by the MFH KG on dietary recommendation tasks. Overall, the integration of the MFH KG substantially improved model performance across all evaluated metrics, with an average increase of 14.5% in Hits@1 and 8.7% in F₁-score, respectively. Among the baseline LLMs, DeepSeek-R1 achieved the best performance (Hits@1: 70.4%, F₁-score: 64.1%), followed by GPT-4 (Hits@1: 68.6%, F₁-score: 55.8%), Qwen2.5-7B (Hits@1: 58.7%, F₁-score: 53.2%), and LLaMA2-Chat-7B (Hits@1: 54.1%, F₁-score: 49.3%). The performance ranking remained consistent after MFH KG augmentation, with DeepSeek-R1 outperforming the others (Hits@1: 84.2%, F₁-score: 71.5%). Due to its superior performance in the above automated evaluation, DeepSeek-R1 was selected as the LLM for all subsequent experiments.

In human evaluation, we conducted a comparative assessment between LLMs and MFH KG–augmented LLMs across 5 key metrics. As shown in Table 3, the MFH KG–augmented DeepSeek-R1 consistently outperformed the baseline DeepSeek-R1 model across all evaluation criteria. The MFH KG–augmented model achieved substantially higher scores in rationality (4.1 vs 3.2), explainability (4.3 vs 2.9), user acceptability (4.4 vs 3.6), personalization (4.2 vs 3.3), and consistency (4.6 vs 3.4). These results provide compelling evidence that augmenting DeepSeek-R1 with the specialized MFH KG significantly improves the quality of dietary recommendations based on MFH principles.

We analyzed the impact of parameter $k$, which controls the number of retrieved reasoning paths in the RAG module. The evaluation across $k$ values [1, 3, 5, 10, 20, and 30] is shown in Figure 3. The F₁-scores indicate that small increases in $k$ initially improved performance, but after reaching an optimal point ($k$=10), further increases led to gradual accuracy decline. This pattern demonstrates the effectiveness of our reasoning path postprocessing, as it successfully prioritizes the most relevant reasoning paths while filtering out potentially misleading or irrelevant information.

‎

To study the impact of the damping factor $d$, where 1-d is the probability of randomly jumping to other nodes, we varied $d$ within the range of [0.1, 0.9]. As shown in Figure 4, the performance of Yaoshi-RAG initially improved with increasing $d$, reached the peak ($d$=0.8), and subsequently declined. This trend aligns with the commonly adopted value ($d$=0.85) proposed by Page et al [47] and is consistent with previous findings [48,49]. Specifically, when $d$ is too small, the resulting PageRank distribution is dominated by random jumps, which weakens the influence of the graph structure and yields less meaningful entity rankings. Conversely, when $d$ approaches 1, the distribution becomes nearly uniform, and the convergence of the power iteration method slows significantly, posing computational challenges.

‎

We evaluated the effectiveness of Yaoshi-RAG’s retrieval strategy against multiple baselines. For sparse retrieval, we used BM25 [50] and followed DecAF [51] by transforming 1-hop subgraphs of topic entities into text. In KG embedding–based methods (TransE [52] RAG and RotatE [53] RAG), we ranked triples by embedding similarity. We also extracted the 2-hop subgraph to prioritize semantically relevant triples from query entities’ neighborhood. All baseline methods retained the top 30 triples as retrieval results. Table 4 shows the performance comparison of different retrieval strategies. Yaoshi-RAG achieved superior performance with the highest Hits@1 (84.2%) and F₁-score (71.5%).

To evaluate the impact of key modules in Yaoshi-RAG, we compared three variants: (1) a variant replacing the UKG with a deterministic KG, in which all triples were assumed to be fully reliable (ie, confidence scores were set to 1); (2) a variant removing the postprocessing of retrieval results, in which all retrieved reasoning paths from the UKG were directly input into the LLM without ranking or filtering; and (3) a variant randomly generating relation paths to guide the retrieval of reasoning paths.

As shown in Table 5, using a deterministic KG leads to a performance decline, highlighting the importance of incorporating uncertainty information derived from the UKG. Performance degrades further when postprocessing is removed, due to the inclusion of noisy and irrelevant reasoning paths. This confirms the effectiveness of leveraging weighted PageRank and triple confidence scores in identifying relevant reasoning paths. The variant using randomly generated relation paths for retrieval performs the worst among all settings, emphasizing the necessity of a retrieval strategy that generates faithful relation paths.

Textbox 5 demonstrates an actual case of a dietary question and the answer generated by our proposed framework. The framework delivers a comprehensive response to a patient presenting peripheral heat sensations and sleep disturbances. By leveraging reasoning paths derived from the MFH KG, the framework establishes clear relations between reported symptoms, underlying constitutional patterns, and appropriate dietary recommendations. The framework also enriches its recommendations by incorporating detailed food entities’ attributes from the KG into the LLM, thereby enhancing the specificity of guidance regarding consumption methods and other relevant details. This case demonstrates how Yaoshi-RAG effectively translates MFH principles into clear and practical guidance, helping patients better understand the recommendations and adhere to the suggested treatments.

Discussion

Principal Findings

Evaluation results indicate that our framework improved the performance of all evaluated LLMs, showcasing its robust adaptability. High-performing models, such as DeepSeek-R1, exhibited marked improvements when enhanced by the MFH KG. This suggests that the MFH KG offers valuable context about ingredient relations, therapeutic properties, and dietary principles, which might not be fully captured during the models’ pretraining. Remarkably, even LLMs with fewer parameters showed performance improvement when integrated with the MFH KG. These results highlight the effectiveness of the proposed framework, which combines the contextual reasoning capabilities of LLMs with the rich and structured knowledge of KGs, making it a powerful tool for MFH-based dietary recommendations.

The human evaluation results revealed that the KG-augmented LLM demonstrated comprehensive improvements. The augmented LLM exhibited improved rationality through the relations between symptoms and dietary recommendations provided by the MFH KG. It also achieved superior explainability by integrating theoretical reasoning paths. Furthermore, the framework demonstrated increased user acceptability with more clinically applicable suggestions. The augmented LLM also maintained greater consistency across varying inputs. These findings collectively indicate that RAG can effectively enhance LLMs’ capabilities with domain-specific knowledge.

The parameter analysis and ablation studies validate the effectiveness of our postprocessing strategy applied to the retrieved reasoning paths. Retrieving more information does not necessarily lead to performance improvement and may instead introduce irrelevant or noisy information [54], thereby degrading the quality of recommendations. By incorporating triple confidence scores from UKG reasoning, the framework prioritizes informative reasoning paths for dietary recommendations. The performance decline beyond $k$=10 indicates that confidence-based ranking effectively distinguishes informative reasoning paths from less relevant ones. This underscores the importance of uncertainty modeling in TCM knowledge representation, particularly given the heterogeneous quality of evidence. By measuring confidence scores, the inherent uncertainty in MFH principles can be better managed, prioritizing reliable relations while mitigating the influence of less trustworthy ones.

Limitations

Several limitations of this study should be acknowledged. The MFH KG was constructed using OpenIE, which may introduce inaccuracies and cannot cover all knowledge on MFH principles. Besides, we did not fine-tune the LLMs specifically for relation path generation, which may have constrained the retrieval effectiveness of our framework, particularly for complex cases involving multiple symptoms or conditions. Addressing these limitations in the future would likely further enhance the LLMs’ clinical uses and recommendation quality.

Conclusions

This study shows Yaoshi-RAG, a new framework that enhances LLMs’ capabilities in generating MFH dietary recommendations through the integration of a UKG. This framework uses multistep LLM calls for OpenIE to automatically construct an MFH KG, incorporating UKG reasoning to measure the confidence of existing triples and complete missing triples. Upon receiving user queries, the framework implements a reasoning path–based knowledge retrieval strategy, extracting relevant reasoning paths from the MFH KG and optimizing the retrieval results through a postprocessing mechanism. Experimental evaluations demonstrate that DeepSeek-R1 is the best-performing base model for MFH-based dietary recommendation generation. This framework facilitates dietary recommendations that adapt to individual health conditions and symptom requirements while considering diverse ingredients and dishes, accompanied by comprehensive explanations grounded in MFH principles. In the future, we plan to focus on implementing more effective knowledge extraction methodologies, expanding our KG through additional MFH literature, exploring optimized retrieval strategies, and fine-tuning open-source LLMs to further improve the accuracy and reliability of the generated dietary recommendations.