[论文解读] Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning
RoG 将知识图谱与大型语言模型结合，在计划-检索-推理框架中实现可信、可解释的 KG 推理和最先进的 KGQA 性能。它将 KG 知识蒸馏到 LLM，并检索 KG 路径以指导推理。
Large language models (LLMs) have demonstrated impressive reasoning abilities in complex tasks. However, they lack up-to-date knowledge and experience hallucinations during reasoning, which can lead to incorrect reasoning processes and diminish their performance and trustworthiness. Knowledge graphs (KGs), which capture vast amounts of facts in a structured format, offer a reliable source of knowledge for reasoning. Nevertheless, existing KG-based LLM reasoning methods only treat KGs as factual knowledge bases and overlook the importance of their structural information for reasoning. In this paper, we propose a novel method called reasoning on graphs (RoG) that synergizes LLMs with KGs to enable faithful and interpretable reasoning. Specifically, we present a planning-retrieval-reasoning framework, where RoG first generates relation paths grounded by KGs as faithful plans. These plans are then used to retrieve valid reasoning paths from the KGs for LLMs to conduct faithful reasoning. Furthermore, RoG not only distills knowledge from KGs to improve the reasoning ability of LLMs through training but also allows seamless integration with any arbitrary LLMs during inference. Extensive experiments on two benchmark KGQA datasets demonstrate that RoG achieves state-of-the-art performance on KG reasoning tasks and generates faithful and interpretable reasoning results.
研究动机与目标
- 以利用结构化 KG 知识来推动 LLM 的可信推理。
- 提出一个将 LLM 计划基于 KG 结构进行支撑的计划-检索-推理框架。
- 将 KG 知识蒸馏到 LLM，并在推理阶段实现与任意 LLM 的集成。
- 在基准数据集上展示最先进的 KGQA 性能和可解释的推理。]
- method':['生成以 KG 为依据的关系路径，作为对 LLM 的可信计划提示。','从 KG 中检索遵循计划关系路径的推理路径。','使用推理模块基于检索到的路径给出可解释说明的答案。','通过两项指令微调任务进行优化：计划优化和检索-推理优化。','将目标设定为最大化给定问题和检索路径下正确答案的概率（基于 ELBO 的优化）。','在推理阶段允许与任意 LLM 的即插即用集成。']
- research_questions':['RoG 是否能够在 KGQA 任务上达到最先进的性能？','RoG 的计划模块是否可以与其他 LLM 集成以提升性能？','RoG 是否可以对其他知识图谱进行微调和迁移？','RoG 是否能够提供可信的推理和可解释的结果？']
- key_findings':['RoG 在 WebQSP 和 CWQ KGQA 基准测试上达到最先进的性能。','将 RoG 的计划与多种 LLM 集成可显著提升它们的推理准确性。','RoG 展示了对另一个 KG（MetaQA-3hop）的可迁移性，并具备有效的微调策略。','RoG 提供可信的推理路径和可解释的解释，相对于基线减少了幻觉。','消融研究表明计划模块和推理模块对性能提升至关重要。']
- table_headers':['类型','方法','Hits@1 (WebQSP)','F1 (WebQSP)','Hits@1 (CWQ)','F1 (CWQ)'],
- table_rows:[["Embedding","KV-Mem","46.7","34.5","18.4","15.7"],["Embedding","EmbedKGQA","66.6","-","45.9","-"],["Embedding","NSM","68.7","62.8","47.6","42.4"],["Embedding","TransferNet","71.4","-","48.6","-"],["Embedding","KGT5","56.1","-","36.5","-"],["Retrieval","GraftNet","66.4","60.4","36.8","32.7"],["Retrieval","PullNet","68.1","-","45.9","-"],["Retrieval","SR+NSM","68.9","64.1","50.2","47.1"],["Retrieval","SR+NSM+E2E","69.5","64.1","49.3","46.3"],["Semantic Parsing","SPARQL","-","-","31.6","-"],["Semantic Parsing","QGG","73.0","73.8","36.9","37.4"],["Semantic Parsing","ArcaneQA","-","75.3","-","-"],["Semantic Parsing","RnG-KBQA","-","76.2","-","-"],["LLMs","Flan-T5-xl","31.0","-","14.7","-"],["LLMs","Alpaca-7B","51.8","-","27.4","-"],["LLMs","LLaMA2-Chat-7B","64.4","-","34.6","-"],["LLMs+KGs","KD-CoT","68.6","52.5","55.7","-"],["LLMs+KGs","UniKGQA","77.2","72.2","51.2","49.1"],["LLMs+KGs","DECAF (DPR+FiD-3B)","82.1","78.8","-","-"],["RoG","RoG","85.7","70.8","62.6","56.2"]]}]}{
- note: 该 JSON 为示例输出格式，实际应以标准 JSON 编码提供，确保表格字段和内容严格保持原样数值不变。
更好的研究，从现在开始
从论文设计到论文写作，大幅缩短您的研究时间。
无需绑定信用卡
本解读由 AI 生成，并经人工编辑审核。