[论文解读] Improving Medical Reasoning through Retrieval and Self-Reflection with Retrieval-Augmented Large Language Models

Minbyul Jeong, Jiwoong Sohn+2 more|arXiv (Cornell University)|Jan 27, 2024

Topic Modeling被引用 4

一句话总结

Self-BioRAG 是一种专为生物医学和临床问答任务设计的检索增强型、自我反思式大语言模型框架。它通过动态决定何时检索特定领域的证据、将其与参数化知识整合，并自我评估推理过程，从而提升医学推理能力；在三个主要医学基准测试中，相较于最先进的 7B 开源模型，其准确率绝对提升了 7.2%。

ABSTRACT

Recent proprietary large language models (LLMs), such as GPT-4, have achieved a milestone in tackling diverse challenges in the biomedical domain, ranging from multiple-choice questions to long-form generations. To address challenges that still cannot be handled with the encoded knowledge of LLMs, various retrieval-augmented generation (RAG) methods have been developed by searching documents from the knowledge corpus and appending them unconditionally or selectively to the input of LLMs for generation. However, when applying existing methods to different domain-specific problems, poor generalization becomes apparent, leading to fetching incorrect documents or making inaccurate judgments. In this paper, we introduce Self-BioRAG, a framework reliable for biomedical text that specializes in generating explanations, retrieving domain-specific documents, and self-reflecting generated responses. We utilize 84k filtered biomedical instruction sets to train Self-BioRAG that can assess its generated explanations with customized reflective tokens. Our work proves that domain-specific components, such as a retriever, domain-related document corpus, and instruction sets are necessary for adhering to domain-related instructions. Using three major medical question-answering benchmark datasets, experimental results of Self-BioRAG demonstrate significant performance gains by achieving a 7.2% absolute improvement on average over the state-of-the-art open-foundation model with a parameter size of 7B or less. Overall, we analyze that Self-BioRAG finds the clues in the question, retrieves relevant documents if needed, and understands how to answer with information from retrieved documents and encoded knowledge as a medical expert does. We release our data and code for training our framework components and model weights (7B and 13B) to enhance capabilities in biomedical and clinical domains.

研究动机与目标

• 为解决通用大语言模型在生物医学推理中的局限性，特别是幻觉现象和事实基础薄弱的问题。
• 通过将按需检索与自我反思能力相结合，提升特定领域医学问答任务的性能。
• 证明领域特定组件（如生物医学检索器、精选语料库和指令微调模型）对于实现高保真医学推理至关重要。
• 开发一种框架，通过结合检索到的证据与参数化知识并辅以自我评估，模拟专家医学推理过程。
• 发布经过完整训练的 7B 和 13B 模型，以及相关数据与代码，以推动临床与生物医学大语言模型应用的研究。

提出的方法

• 该框架使用在 84,000 个过滤后的生物医学指令数据集上微调的领域特定指令微调大语言模型（7B/13B），用于生成答案并进行自我反思。
• 它采用检索器（MedCPT）从精选语料库中获取相关生物医学文献：PubMed 摘要、PMC 全文、临床指南和医学教科书。
• Self-BioRAG 使用反思性标记（tokens）自我评估是否需要检索、检索到的证据是否支持答案，以及最终回答是否具有实用性。
• 模型根据问题复杂度和证据需求，动态决定是直接使用参数化知识生成答案，还是采用检索增强生成。
• 它将检索到的证据（以蓝色突出显示）与模型内部知识（以红色突出显示）相结合，生成推理过程与最终答案。
• 该框架以端到端方式训练，重点聚焦于多选题问答、摘要生成和临床推理任务。

Figure 1 : Comparison between three frameworks: generation using language model (LM), retrieval-augmented generation (RAG) using LM, and our Self-BioRAG. (A) depicts the process of sequence-to-sequence generation of LM. (B) The RAG framework first finds relevant documents from large-scale corpus suc

实验结果

研究问题

• RQ1 检索增强型、自我反思式大语言模型框架是否能在生物医学问答任务中超越通用大模型？
• RQ2 使用领域特定组件——包括检索器、语料库、指令数据和反思机制——是否能显著提升推理准确率？
• RQ3 自我反思标记是否能有效指导医学情境下的检索决策与答案质量？
• RQ4 在医学推理中，结合检索证据与参数化知识与仅依赖内部知识相比，表现如何？
• RQ5 该框架是否能泛化到不同类型医学问题，包括需要临床指南或教科书知识的问题？

主要发现

• Self-BioRAG 在三个主要医学问答基准测试中，平均准确率相较于最先进的开源 7B 模型绝对提升了 7.2%。
• 在 PubMedQA 数据集上，Self-BioRAG 达到 54.6% 的准确率，略低于 LLaMA2（56.0%）和 MEDITRON（58.1%），归因于检索过程中证据存在噪声。
• 在解答 USMLE 风格问题时，该模型从医学教科书语料库中检索到显著更高比例的相关证据。
• Self-BioRAG 能够正确识别何时无需检索，对简单问题仅依赖参数化知识；对复杂且依赖证据的问题则启用检索。
• 结合检索证据与参数化知识可产生更准确且可解释的诊断结果，如在多囊卵巢综合征（PCOS）和糖耐量受损病例研究中所展示。
• 该框架表明，领域特定组件至关重要，因为通用大语言模型检索增强方法在临床与生物医学场景中难以有效泛化。

Figure 2 : Overview of our Self-BioRAG process: data construction, training, and inference of Self-Reflection Language Model (critic LM $C$ ) and Domain-specific Instruction-tuned Language Model (generator LM $M$ ). We construct 120k biomedical instruction sets using two off-the-shelf instruction se