This study used a parallel 2‑arm Delphi design. In total, 50 international panelists—including clinicians, researchers, and representatives of patient organizations—were recruited via professional networks, stratified, and randomized into 2 groups of 25 (50%) participants each to ensure a balanced distribution of professional backgrounds across the 2 arms (arm A: human revision; arm B: AI‑assisted revision). Before participation, panelists were informed that the study included both a human revision arm and an AI-assisted revision arm; however, they were not informed of their individual allocation and, therefore, did not know which revision process had been applied to the statements circulated within their arm. Panelists assessed 28 statements concerning the development, studies related to quality, and preclinical aspects and regulatory issues of the classification of RNA‑based medicines. Statements were rated on a 5‑point Likert scale (1=“strongly disagree”; 5=“strongly agree”). A consensus threshold was predefined as 75% or more of the panelists selecting “agree” or “strongly agree.” Round 1 remained open for 2 weeks. After closure, statements below the threshold entered the revision process. Operational end points such as turnaround time, reviewer workload, and cost were not prospectively measured as the study was designed to compare consensus outcomes after revision between arms. Panelists in both arms evaluated the same set of statements using the same 5-point Likert scale within the same 2-week response window; accordingly, panelist time commitment was intended to be comparable across arms.

The AI pipeline (Figure 1) was designed to replicate five key steps undertaken by human reviewers: (1) analyzing panelist comments to detect requests for clarification or additional evidence, (2) identifying external references suggested by panelists, (3) summarizing supporting PDF documents, (4) retrieving text segments relevant to the statement, and (5) drafting a revised statement with a transparent rationale and explicit citations. Three specialized agents formed the core of the pipeline:

To ground revisions in verifiable evidence, the pipeline used a hybrid retrieval module. Text from the supporting PDFs was segmented and indexed using both dense embeddings (all‑MiniLM‑L6‑v2) and sparse BM25 representations.

The retrieval system operated on a static, curated source library composed of the supporting PDF documents available during the Delphi process together with any additional references suggested by panelists and validated by human reviewers before inclusion. The pipeline did not access regulatory databases or bibliographic sources in real time.

For each query (the subthreshold statement), both retrievers returned a fixed number of candidate passages. A cross‑encoder (ms‑marco‑MiniLM‑L‑6‑v2) reranked the combined candidates selecting the most relevant excerpts. During the hybrid fusion step, the final retrieval score was computed as a weighted combination of the individual retrievers, assigning a weight of 0.7 to the dense semantic scores and 0.3 to the sparse lexical (BM25) scores to favor semantic depth while retaining lexical coverage. Hybrid retrieval improves recall and precision compared with either dense or sparse retrieval alone; RAG combines pretrained sequence‑to‑sequence models with a dense vector index to generate more specific and factual answers [14]. Hybrid systems that fuse sparse BM25 and neural embeddings outperform either method individually in text retrieval tasks [7], and knowledge distillation from cross‑encoder rerankers yields dense representations that retain much of the accuracy of cross‑encoders while reducing storage requirements; when combined with sparse signals, these distilled embeddings approach cross‑encoder performance [8]. In biomedical search, hybrid first‑stage retrieval models combining lexical and neural methods consistently surpass purely sparse or purely neural models [15], and hybrid reranking schemes that aggregate similarity scores from dense and sparse retrievers deliver stable improvements over baseline rerankers [16]. We implemented our vector store using the Faiss library, which offers efficient algorithms for searching, clustering, and compressing vector representations [17]. Emerging work explores resource‑constrained hybrid retrieval architectures leveraging smaller language models to support RAG on edge devices, although these approaches were not tested in our pipeline [18].

At each stage, oversight was provided by 2 researchers with expertise in Delphi methodology whose familiarity with the topic derived from their involvement in the development of the statements through literature review. They were not panelists. In the AI-assisted arm, their role was limited to verification of retrieved references, summaries, and draft revisions rather than full manual interpretation of panelist comments and complete redrafting of statements. After reference detection and summarization, they confirmed that the references and summaries were appropriate. After retrieval and refinement, they examined the revised statement, change log, and citations. If a revision contained errors or omissions, they could request additional iterations or edit the statement manually. Reviewer time and cost were not prospectively recorded. This human‑in‑the‑loop design aligns with recommendations that LLM outputs be verified and that AI tools augment rather than replace expert judgment [19].

For each statement and round, the percentage of panelists selecting “agree” or “strongly agree” was calculated. Statements were classified as “passed” or “failed” based on the 75% or more threshold. We compared the number and proportion of statements passing in each arm across rounds using descriptive statistics. Because panelists evaluated the same statements across arms, no inferential tests were performed. Improvements were expressed as percentage‑point differences between rounds.

This methodological study involved an anonymous expert elicitation through a Delphi consensus procedure to evaluate aggregate statements on RNA-based medicines. Per the guidelines of local and institutional ethics committees regarding noninterventional research, formal ethics approval was not required as the study did not process individual patient health data, collect sensitive personal parameters, or involve clinical interventions on human participants. All participants were international health care experts, clinicians, and organizational representatives who took part voluntarily and were fully informed about the study’s scope, objectives, and data use protocols prior to completing the questionnaires. Completing the online feedback was considered informed consent to the use of aggregated, anonymized responses.

This controlled study demonstrates that a citation‑anchored, multi‑agent AI pipeline can nearly match human reviewers in rescuing subthreshold Delphi statements. The AI arm started from a lower baseline but achieved consensus on 85.7% (24/28) of the statements after 1 revision cycle, closing most of the gap with the human arm. AI‑assisted revisions were particularly effective for statements far below the threshold, suggesting that comprehensive retrieval and structured reasoning can surface evidence and clarify ambiguities that panelists may have overlooked. This is in line with recent AI-assisted post hoc assessment of published expert consensus showing that LLMs may help highlight convergent interpretations and identify statements deserving closer evidentiary review when evidence is limited while not replacing independent expert judgment [20]. Residual differences between arms were modest and may reflect panel heterogeneity rather than intrinsic limitations of the AI.

Our findings align with a growing body of evidence that RAG and hybrid retrieval substantially improve the reliability of LLM outputs. RAG couples a pretrained sequence‑to‑sequence model with a dense vector index so that the generation step can draw on external documents and, thus, produce more specific and factual answers [14]. Hybrid systems that fuse semantic (dense) and lexical (sparse) signals achieve superior recall and ranking precision compared with either method alone [7,8]. These results informed our design of a dense-sparse retrieval module and cross‑encoder reranking.

At the same time, unaugmented LLMs are prone to hallucinations. Adversarial evaluations of GPT‑4 and related models report hallucination rates ranging from 50% to 82%, and prompt engineering mitigations reduce hallucinations only modestly [19]. Such vulnerability underscores the need for retrieval mechanisms and human verification. Multi‑agent frameworks that break down tasks into retrieval, reasoning, and verification phases are increasingly advocated for to mitigate hallucinations and maintain factuality [6]. Our pipeline embodies this “retrieve-summarize-verify” paradigm: it anchors statements to cited evidence, exposes reasoning steps, and allows human reviewers to approve or amend each stage.

The inclusion of human reviewers is not merely precautionary; it reflects emerging guidance for transparent and accountable AI-assisted scientific writing and health-related AI use. In high-stakes consensus settings, AI can assist with retrieval, summarization, and draft revision, but decisions about the relevance of the retrieved evidence, its interpretation and grading, and the final normative framing of recommendations should remain under human responsibility. Reviewers, therefore, validate extracted references, approve summaries, and ensure that revised statements align with domain knowledge and the intended consensus context. This iterative oversight balances the efficiency gains of automation with the trustworthiness of expert judgment and is better understood as collaborative intelligence than as autonomous AI decision-making [19,21-23].

In the context of consensus methodology, our results corroborate the observation that panel composition profoundly influences consensus outcomes. Heterogeneous panels bring diverse perspectives but make consensus harder to achieve [2]. In our study, the AI arm panelists were more critical at baseline, as reflected by both a lower mean Likert-scale score and a lower proportion of statements meeting the consensus threshold in round 1. This likely resulted in more subthreshold statements entering the revision phase and, consequently, a larger apparent improvement after revision. A further probable explanation is residual heterogeneity in panel composition, which is known to influence Delphi outcomes by shaping how conservatively or permissively statements are judged. When these baseline differences are taken into account, the final consensus rates between arms were nearly identical. Our work, therefore, supports the feasibility of integrating AI tools into Delphi processes while underscoring the continued importance of careful panel selection and human expertise.

This study has several limitations. First, the AI pipeline used a specific LLM and retrieval configuration based on a static, curated knowledge base rather than real-time access to regulatory or bibliographic databases; performance may vary with different models, temperatures, retrieval strategies, or source update policies. While live access might improve the timeliness of revisions by incorporating newly published evidence or updated regulatory documents, the use of a static source library improved reproducibility, traceability, and consistency across statements in this controlled study. Second, operational efficiency was not formally quantified. We did not prospectively measure turnaround time, reviewer workload, personnel cost, or the time required from panelists in each arm. Although panelist participation was designed to be comparable across arms and the AI-assisted workflow shifted expert input from full manual rewriting toward verification and minor postediting, this study cannot make formal claims about time or cost savings. Future studies should prospectively compare time to revision, reviewer effort per statement, and cost-effectiveness between human-only and AI-assisted workflows. Third, the human and AI arms differed in baseline criticality, which partly explains the larger improvement in the AI arm. Fourth, our pipeline used a single model; multi‑model ensembles might further reduce variance but would increase computational costs.

Research should extend this evaluation across diverse domains, panel sizes, and consensus thresholds. Comparative studies exploring alternative retrieval strategies, prompt designs, agent configurations, and multi‑model ensembles could identify optimal architectures. Formal assessments of time savings, computational costs, and environmental impact will inform the practical viability of AI‑augmented consensus workflows. Developing automated metrics for citation fidelity and hallucination detection could enhance trust in AI outputs. Qualitative research into panelists’ perceptions of AI‑assisted revisions and their willingness to disclose AI involvement will help refine human‑in‑the‑loop designs and support responsible adoption. In the rapidly evolving field of messenger RNA therapeutics, AI‑assisted consensus methods may help regulators and clinicians keep pace with scientific advances.

This study demonstrates that a modular, retrieval‑augmented AI pipeline can deliver revision outcomes comparable to those of human reviewers in a Delphi consensus exercise on RNA therapeutics. The pipeline rescued a greater proportion of subthreshold statements by aggregating panelist comments, retrieving relevant evidence, and producing citation‑anchored revisions. Human oversight remains essential: experts must verify references, validate summaries, and approve final wording. With appropriate safeguards, AI‑augmented workflows can accelerate consensus development in rapidly evolving fields.