[논문 리뷰] Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering
본 논문은 Retrieve-Rewrite-Answer를 도입합니다. 이는 answer-sensitive KG-to-Text rewrite 모듈을 사용하여 retrieved subgraphs를 informative textual knowledge로 변환하고, 벤치마크 전반에서 KGQA 성능을 개선하는 KG-to-Text 보강 프레임워크입니다.
Despite their competitive performance on knowledge-intensive tasks, large language models (LLMs) still have limitations in memorizing all world knowledge especially long tail knowledge. In this paper, we study the KG-augmented language model approach for solving the knowledge graph question answering (KGQA) task that requires rich world knowledge. Existing work has shown that retrieving KG knowledge to enhance LLMs prompting can significantly improve LLMs performance in KGQA. However, their approaches lack a well-formed verbalization of KG knowledge, i.e., they ignore the gap between KG representations and textual representations. To this end, we propose an answer-sensitive KG-to-Text approach that can transform KG knowledge into well-textualized statements most informative for KGQA. Based on this approach, we propose a KG-to-Text enhanced LLMs framework for solving the KGQA task. Experiments on several KGQA benchmarks show that the proposed KG-to-Text augmented LLMs approach outperforms previous KG-augmented LLMs approaches regarding answer accuracy and usefulness of knowledge statements.
연구 동기 및 목표
- Motivate the use of KG knowledge to address knowledge gaps in LLMs for KGQA.
- Propose a three-stage Retrieve-Rewrite-Answer framework to improve answer accuracy.
- Address scarcity of KG-to-Text labeled data via automatic corpus generation guided by QA feedback.
- Show that textual knowledge representations outperform plain triples for KGQA across multiple LLMs and benchmarks.
제안 방법
- Retrieve: predict hop count, predict relation paths, and sample KG triples to form subgraphs.
- Rewrite: fine-tune open-source LLMs on a KG-to-Text corpus to convert subgraphs into coherent, answer-relevant text.
- Answer: integrate the generated textual knowledge with prompts to a QA model to produce the final answer.
- Corpus generation: automatically create graph-text pairs by extracting subgraphs from QA benchmarks and generating text with ChatGPT, guided by QA feedback to ensure usefulness of knowledge for KGQA.
- Evaluation setup: compare KG-to-Text augmented LLMs against baselines on multiple KGQA benchmarks using various LLMs and representation formats.]
- research_questions: ["Can answer-sensitive KG-to-Text textualizations improve KGQA accuracy over triple-based or no-knowledge baselines?", "How does knowledge representation format (triple vs. free-form text vs. other augmented formats) impact KGQA performance across different LLMs?", "Does automatic KG-to-Text corpus generation yield beneficial fine-tuning data for KG-to-Text models in KGQA contexts?", "How do different LLMs (size and architecture) respond to the proposed KG-to-Text augmented prompts in KGQA tasks?"]
- key_findings: ["The proposed Retrieve-Rewrite-Answer framework outperforms prior KG-augmented LLM approaches across several LLM backbones and benchmarks.", " Free-form KG-to-Text knowledge generally provides greater benefits for KGQA than raw triples, indicating the value of semantically coherent textualization.", " Fine-tuning open-source LLMs on a KG-to-Text corpus yields informative knowledge descriptions that enhance QA performance.", " Knowledge representation format significantly influences KGQA outcomes, with answer-sensitive textual knowledge outperforming other representations across multiple models.", " The approach demonstrates robustness across diverse KGQA benchmarks (including MetaQA, WebQSP/WebQ) and multilingual settings (e.g., Chinese ZJQA)."],
- table_headers: []
- table_rows: []
더 나은 연구,지금 바로 시작하세요
연구 설계부터 논문 작성까지, 연구 시간을 획기적으로 줄여보세요.
카드 등록 없음 · 무료 플랜 제공
이 리뷰는 AI가 만들고, 인간 에디터가 검토했습니다.