[Paper Review] Multiview Identifiers Enhanced Generative Retrieval

Yongqi Li, Nan Yang+3 more|arXiv (Cornell University)|May 26, 2023

Topic Modeling4 citations

TL;DR

This paper proposes MINDER, a multiview identifiers enhanced generative retrieval framework that generates synthetic identifiers—specifically pseudo-queries—from passage content to improve retrieval effectiveness. By jointly leveraging titles, substrings, and content-based pseudo-queries as complementary views, MINDER achieves state-of-the-art performance on three public datasets, demonstrating superior robustness and coverage across diverse query types.

ABSTRACT

Instead of simply matching a query to pre-existing passages, generative retrieval generates identifier strings of passages as the retrieval target. At a cost, the identifier must be distinctive enough to represent a passage. Current approaches use either a numeric ID or a text piece (such as a title or substrings) as the identifier. However, these identifiers cannot cover a passage's content well. As such, we are motivated to propose a new type of identifier, synthetic identifiers, that are generated based on the content of a passage and could integrate contextualized information that text pieces lack. Furthermore, we simultaneously consider multiview identifiers, including synthetic identifiers, titles, and substrings. These views of identifiers complement each other and facilitate the holistic ranking of passages from multiple perspectives. We conduct a series of experiments on three public datasets, and the results indicate that our proposed approach performs the best in generative retrieval, demonstrating its effectiveness and robustness.

Motivation & Objective

• To address the limitation of existing generative retrieval methods that rely on static identifiers like titles or numeric IDs, which lack contextual richness and holistic coverage.
• To improve retrieval performance by introducing synthetic identifiers derived from passage content, specifically pseudo-queries that capture multi-sentence semantics.
• To enable holistic passage ranking by simultaneously considering multiple views of identifiers—titles, substrings, and pseudo-queries—each suited to different query types.
• To demonstrate that multiview identifier integration enhances model robustness and effectiveness across diverse retrieval scenarios.

Proposed method

• MINDER generates identifiers using an autoregressive language model conditioned on query and identifier type (e.g., 'title', 'substring', 'pseudo-query').
• Synthetic identifiers are created by generating question-like rephrasings of passage segments, which serve as contextually rich, multi-sentence identifiers.
• The model employs beam search to produce multiple candidate identifiers per view, which are then aggregated via a heuristic function to rank candidate passages.
• Passages are ranked based on their coverage across all three identifier views, enabling complementary scoring from different perspectives.
• A FM-index data structure is used to efficiently store and retrieve identifiers, ensuring linear space complexity relative to identifier size.

Experimental results

Research questions

• RQ1 Can synthetic identifiers derived from passage content improve generative retrieval beyond static identifiers like titles or substrings?
• RQ2 How does combining multiple views of identifiers (title, substring, pseudo-query) enhance retrieval performance compared to single-view approaches?
• RQ3 To what extent do multiview identifiers improve robustness across diverse query types, including general, detailed, and complex queries?
• RQ4 How does the inclusion of pseudo-queries as synthetic identifiers affect retrieval coverage and accuracy?

Key findings

• MINDER achieves state-of-the-art performance on three public datasets—MSMARCO, NQ, and TriviaQA—outperforming existing generative retrieval methods.
• The inclusion of pseudo-queries as synthetic identifiers significantly improves retrieval coverage, especially for complex queries requiring multi-sentence understanding.
• Beam size of 15–20 yields optimal performance on both NQ and TriviaQA, though a smaller beam size (e.g., 5) is preferable on MSMARCO due to high passage redundancy.
• The substring view contributes less effectively on MSMARCO due to low distinctiveness, while the title and pseudo-query views show stronger performance across datasets.
• MINDER requires 1.2× more inference time than SEAL due to the multiview generation process, but this is offset by improved retrieval accuracy.