[论文解读] Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering

Rajarshi Das, Shehzaad Dhuliawala+2 more|arXiv (Cornell University)|May 14, 2019

Topic Modeling被引用 35

一句话总结

引入一个开放域问答框架，其中一个快速检索器与一个神经阅读器通过一个多步推理器进行迭代交互，从而改写查询以提高跨多个段落的检索和聚合。

ABSTRACT

This paper introduces a new framework for open-domain question answering in which the retriever and the reader iteratively interact with each other. The framework is agnostic to the architecture of the machine reading model, only requiring access to the token-level hidden representations of the reader. The retriever uses fast nearest neighbor search to scale to corpora containing millions of paragraphs. A gated recurrent unit updates the query at each step conditioned on the state of the reader and the reformulated query is used to re-rank the paragraphs by the retriever. We conduct analysis and show that iterative interaction helps in retrieving informative paragraphs from the corpus. Finally, we show that our multi-step-reasoning framework brings consistent improvement when applied to two widely used reader architectures DrQA and BiDAF on various large open-domain datasets --- TriviaQA-unfiltered, QuasarT, SearchQA, and SQuAD-Open.

研究动机与目标

• Motivate scalable open-domain QA with fast retrieval and interactive reading.
• Propose a three-component architecture (retriever, reader, multi-step-reasoner) that enables iterative query reformulation.
• Demonstrate architecture-agnostic gains by applying to DrQA and BiDAF readers across large datasets.

提出的方法

• Paragraph representations are pre-computed and cached to enable scalable fast inner-product retrieval.
• Retriever ranks top-k paragraphs using inner product between a query vector and pre-computed paragraph vectors.
• Reader (DrQA or BiDAF) processes the top-k paragraphs to extract answer spans and produce a reader state.
• Multi-step-reasoner, a gated recurrent unit, updates the query vector using the reader state to re-rank and fetch new paragraphs (iterative interaction).
• Training uses reinforcement learning to optimize the multi-step-reasoner for better readership performance; the reader is pre-trained with supervised learning, and the multi-step-reasoner is fine-tuned with policy gradients.

实验结果

研究问题

• RQ1 Can iterative interaction between retriever and reader improve open-domain QA performance over large corpora?
• RQ2 Does query reformulation via a multi-step-reasoner help in aggregating evidence from multiple paragraphs?
• RQ3 Is the approach architecture-agnostic with respect to the underlying reader model?
• RQ4 How does the framework scale to millions of paragraphs and large open-domain datasets?

主要发现

• Iterative interaction through the multi-step-reasoner yields consistent improvements across DrQA and BiDAF readers on four open-domain QA datasets.
• The multi-step approach improves retrieval quality, with higher P@k after several steps compared to the initial retriever.
• On TriviaQA-open with 1.6M paragraphs per query, the baseline DrQA (no interaction) scores EM 37.45 and F1 42.16, while with 3 interaction steps it reaches EM 39.76 and F1 44.30.
• Across Quasar-T, SearchQA, TriviaQA-open, and Squad-open, the multi-step-reasoner-enhanced models achieve higher EM and F1 than several baselines, with notable gains over the DrQA baseline.
• The retriever with multi-step interaction scales to millions of paragraphs using inner-product search plus SGTree, maintaining scalability on GPU/CPU and outperforming some larger, more expensive baselines in retrieval timing.