[论文解读] UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graph

Jinhao Jiang, Kun Zhou+2 more|arXiv (Cornell University)|Dec 2, 2022

Topic Modeling被引用 28

一句话总结

UniKGQA 提出一个统一模型，联合处理多跳知识图谱问答的检索与推理，使用语义匹配的 PLM 和信息传播模块，检索使用抽象子图且共享预训练任务。

ABSTRACT

Multi-hop Question Answering over Knowledge Graph~(KGQA) aims to find the answer entities that are multiple hops away from the topic entities mentioned in a natural language question on a large-scale Knowledge Graph (KG). To cope with the vast search space, existing work usually adopts a two-stage approach: it first retrieves a relatively small subgraph related to the question and then performs the reasoning on the subgraph to find the answer entities accurately. Although these two stages are highly related, previous work employs very different technical solutions for developing the retrieval and reasoning models, neglecting their relatedness in task essence. In this paper, we propose UniKGQA, a novel approach for multi-hop KGQA task, by unifying retrieval and reasoning in both model architecture and parameter learning. For model architecture, UniKGQA consists of a semantic matching module based on a pre-trained language model~(PLM) for question-relation semantic matching, and a matching information propagation module to propagate the matching information along the directed edges on KGs. For parameter learning, we design a shared pre-training task based on question-relation matching for both retrieval and reasoning models, and then propose retrieval- and reasoning-oriented fine-tuning strategies. Compared with previous studies, our approach is more unified, tightly relating the retrieval and reasoning stages. Extensive experiments on three benchmark datasets have demonstrated the effectiveness of our method on the multi-hop KGQA task. Our codes and data are publicly available at~\url{https://github.com/RUCAIBox/UniKGQA}.

研究动机与目标

• Motivate and address the inefficiency of separate retrieval and reasoning stages in multi-hop KGQA.
• Propose a unified architecture that shares parameters and signals between retrieval and reasoning.
• Introduce abstract subgraphs to normalize scale differences between stages.
• Design pre-training and fine-tuning strategies to transfer knowledge between stages.
• Demonstrate effectiveness on benchmark datasets and analyze retrieval quality and training impact.

提出的方法

• A dual-module architecture: semantic matching (SM) with a PLM for question–relation relevance, and a matching information propagation (MIP) module that propagates SM signals along KG edges.
• Abstract subgraphs for retrieval: merge tails of triples with the same head-relation prefix to reduce node scale.
• Shared pre-training task (Question-Relation Matching) with contrastive learning to align questions with relevant relations, using shortest paths between topics and answers to define positives.
• Two-stage fine-tuning: Retrieval on abstract subgraphs (RAS) using KL-divergence against ground-truth abstract-node signals; Reasoning on retrieved subgraphs (RRS) initializing from the retrieval model and fine-tuning with KL-divergence against ground-truth tail signals.
• Unified optimization where PLM parameters are shared and can be fixed or per-stage updated (w/ QU vs w/ QU,RU variants).

实验结果

研究问题

• RQ1 Can a unified model architecture improve both retrieval and reasoning in multi-hop KGQA compared to separate-stage approaches?
• RQ2 Does sharing parameters and transferring relevance information between retrieval and reasoning improve overall QA performance?
• RQ3 Do abstract subgraphs effectively bridge the scale gap between retrieval and reasoning stages without sacrificing accuracy?
• RQ4 Can pre-training on question–relation matching and subsequent fine-tuning yield efficient and effective learning for both stages?

主要发现

模型	WebQSP Hits@1	WebQSP F1	CWQ Hits@1	CWQ F1	MetaQA-1 Hits@1	MetaQA-2 Hits@1	MetaQA-3 Hits@1
KV-Mem	46.7	34.5	18.4	15.7	96.2	82.7	48.9
GraftNet	66.4	60.4	36.8	32.7	97.0	94.8	77.7
PullNet	68.1	-	45.9	-	97.0	99.9	91.4
EmbedKGQA	66.6	-	-	-	97.5	98.8	94.8
NSM	68.7	62.8	47.6	42.4	97.1	99.9	98.9
TransferNet	71.4	-	48.6	-	97.5	100	100
SR+NSM	68.9	64.1	50.2	47.1	-	-	-
SR+NSM+E2E	69.5	64.1	49.3	46.3	-	-	-
UniKGQA	75.1	70.2	50.7	48.0	97.5	99.0	99.1
w QU	77.0	71.0	50.9	49.4	97.6	99.9	99.5
w QU,RU	77.2	72.2	51.2	49.0	98.0	99.9	99.9

• UniKGQA outperforms baselines on WebQSP and CWQ, with notable gains in Hits@1.
• Retrieval evaluation shows higher answer coverage with learned retrieval vs heuristic methods at comparable subgraph sizes.
• Ablation studies confirm two training strategies (pre-training and initialization transfer) are both beneficial.
• Updating PLM encoder only for questions can match or exceed updating for both questions and relations, offering efficiency benefits.
• Unified architecture enables effective transfer of relevance information from retrieval to reasoning, improving final QA metrics.
• Two variants (w QU and w QU,RU) provide strong performance with different computational trade-offs.