[论文解读] RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering

Xi Ye, Semih Yavuz+3 more|arXiv (Cornell University)|Sep 17, 2021

Topic Modeling被引用 26

一句话总结

RnG-KBQA 引入一个面向知识库问答的生成增强排序器，将对比学习的 BERT 基排序器与基于 T5 的生成器结合，在 GrailQA 和 WebQSP 上达到最新的SOTA，尤其在零样本/泛化设置下。

ABSTRACT

Existing KBQA approaches, despite achieving strong performance on i.i.d. test data, often struggle in generalizing to questions involving unseen KB schema items. Prior ranking-based approaches have shown some success in generalization, but suffer from the coverage issue. We present RnG-KBQA, a Rank-and-Generate approach for KBQA, which remedies the coverage issue with a generation model while preserving a strong generalization capability. Our approach first uses a contrastive ranker to rank a set of candidate logical forms obtained by searching over the knowledge graph. It then introduces a tailored generation model conditioned on the question and the top-ranked candidates to compose the final logical form. We achieve new state-of-the-art results on GrailQA and WebQSP datasets. In particular, our method surpasses the prior state-of-the-art by a large margin on the GrailQA leaderboard. In addition, RnG-KBQA outperforms all prior approaches on the popular WebQSP benchmark, even including the ones that use the oracle entity linking. The experimental results demonstrate the effectiveness of the interplay between ranking and generation, which leads to the superior performance of our proposed approach across all settings with especially strong improvements in zero-shot generalization.

研究动机与目标

• Address generalization gaps in KBQA, especially for unseen KB schema items and compositions.
• Mitigate ranking coverage issues by pairing ranking with generation to cover broader logical form space.
• Leverage pretrained language models to improve generalization and interpretability of KBQA.

提出的方法

• Enumerate candidate logical forms by two-hop KB path exploration and form s-expressions.
• Train a BERT-based bi-encoder ranker with a contrastive objective to score question–candidate pairs.
• Extend ranking with a generation model (T5) that conditions on the question and top-k ranked candidates to produce the final logical form.
• Use an execution-augmented inference to ensure executability and select valid answers from top-k generated forms.

实验结果

研究问题

• RQ1 Can a generation-augmented approach improve generalization in KBQA for unseen KB schema items and compositions?
• RQ2 Does the interplay between a contrastive ranker and a generator improve coverage beyond ranking-only methods?
• RQ3 How does the model perform on zero-shot and compositional generalization benchmarks like GrailQA and WebQSP?

主要发现

• RnG-KBQA achieves new state-of-the-art on GrailQA with EM 68.8 and F1 74.4, surpassing prior SOTA by a substantial margin.
• RnG-KBQA also achieves SOTA on WebQSP with F1 around 75.7 and EM around 71.1 in reported results.
• The generation component notably improves performance over a rank-only baseline, with gains of about 5.3 F1 on GrailQA and 2.9 F1 on WebQSP in ablations.
• Bootstrapped contrastive training for the ranker and the entity disambiguation extension contribute to improved generalization and disambiguation accuracy.
• Executability is high among top-ranked/generated candidates, and the inference procedure robustly selects valid logical forms.