[Paper Review] Document Expansion by Query Prediction

Rodrigo Nogueira, Wei Yang+2 more|arXiv (Cornell University)|Apr 17, 2019

Topic Modeling27 references210 citations

TL;DR

The paper proposes Doc2query, a neural document-expansion method that predicts queries for each document and expands the document text before indexing, achieving strong results with BM25 and optional re-ranking by BERT.

ABSTRACT

One technique to improve the retrieval effectiveness of a search engine is to expand documents with terms that are related or representative of the documents' content.From the perspective of a question answering system, this might comprise questions the document can potentially answer. Following this observation, we propose a simple method that predicts which queries will be issued for a given document and then expands it with those predictions with a vanilla sequence-to-sequence model, trained using datasets consisting of pairs of query and relevant documents. By combining our method with a highly-effective re-ranking component, we achieve the state of the art in two retrieval tasks. In a latency-critical regime, retrieval results alone (without re-ranking) approach the effectiveness of more computationally expensive neural re-rankers but are much faster.

Motivation & Objective

• Address vocabulary mismatch by enriching document representations rather than query representations.
• Predict plausible queries for each document using a sequence-to-sequence transformer.
• Evaluate document expansion on MS MARCO and TREC-CAR datasets to assess retrieval gains.
• Compare document expansion against query expansion and baseline retrieval.
• Demonstrate that expansion prior to indexing can rival or complement neural re-rankers while improving speed.

Proposed method

• Train a transformer-based seq2seq model on (query, relevant document) pairs to generate queries from document text.
• Truncate documents to 400 tokens and 100-token queries to manage memory.
• Generate 10 predicted queries per document using top-k random sampling.
• Append predicted queries to the document and index with BM25.
• Optionally re-rank retrieved results with BERT to boost final rankings.
• Provide open-source code and trained models for reproduction.

Experimental results

Research questions

• RQ1 Can document expansion via predicted queries improve initial retrieval performance over standard BM25?
• RQ2 Is document expansion more effective than query expansion for long-form documents?
• RQ3 What is the impact of using a re-ranker (e.g., BERT) on top of Doc2query-expanded documents?
• RQ4 How does Doc2query perform on diverse datasets such as MS MARCO and TREC-CAR?
• RQ5 What decoding scheme (beam search vs top-k sampling) yields the best retrieval gains?

Key findings

• Document expansion with Doc2query improves retrieval effectiveness by about 15% on both MS MARCO and TREC-CAR compared to BM25 alone.
• Combining Doc2query with a BERT re-ranker achieves best-known results on TREC-CAR and near state-of-the-art on MS MARCO.
• Doc2query expands documents with a mix of copied and novel terms, providing complementary signals for retrieval.
• RM3 query expansion does not improve performance in these precision-oriented datasets.
• Doc2query expansion adds small latency over BM25 but is still faster than neural re-rankers like Duet v2 with BERT.
• Expansion prior to indexing shifts computational load from query-time inference to document-indexing time.