[Paper Review] Joint Retrieval and Generation Training for Grounded Text Generation.

Yizhe Zhang, Siqi Sun+6 more|arXiv (Cornell University)|May 14, 2021

Topic Modeling41 references10 citations

TL;DR

This paper proposes a joint retrieval-and-generation training framework that enables grounded text generation without requiring parallel retrieval-generation data. By leveraging language model signals to jointly train a retriever and generator using a Mixture-of-Experts ensemble, the model learns to retrieve utility-maximizing documents and generate factually grounded, contextually relevant text in both prose and dialogue tasks.

ABSTRACT

Recent advances in large-scale pre-training such as GPT-3 allow seemingly high quality text to be generated from a given prompt. However, such generation systems often suffer from problems of hallucinated facts, and are not inherently designed to incorporate useful external information. Grounded generation models appear to offer remedies, but their training typically relies on rarely-available parallel data where corresponding information-relevant documents are provided for context. We propose a framework that alleviates this data constraint by jointly training a grounded generator and document retriever on the language model signal. The model learns to reward retrieval of the documents with the highest utility in generation, and attentively combines them using a Mixture-of-Experts (MoE) ensemble to generate follow-on text. We demonstrate that both generator and retriever can take advantage of this joint training and work synergistically to produce more informative and relevant text in both prose and dialogue generation.

Motivation & Objective

• To address the lack of parallel retrieval-generation data in training grounded generation models.
• To reduce hallucinations in large language models by integrating external knowledge retrieval during generation.
• To enable synergistic training of a retriever and generator using only autoregressive language modeling signals.
• To improve factual consistency and relevance in text generation through joint optimization of retrieval and generation components.
• To demonstrate the effectiveness of the framework in both open-ended prose and dialogue generation tasks.

Proposed method

• The framework jointly trains a retriever and generator using only autoregressive language modeling objectives, eliminating the need for parallel retrieval-generation annotations.
• The retriever is trained to retrieve documents that maximize the likelihood of the ground-truth generation, using the language model signal as a proxy for utility.
• The generator employs a Mixture-of-Experts (MoE) architecture to dynamically attend to and combine multiple retrieved documents for generation.
• The model uses a single unified objective that optimizes both retrieval and generation simultaneously through masked language modeling.
• The framework enables end-to-end training where retrieval quality directly improves generation fluency and factual accuracy.
• The method leverages the language model's autoregressive capacity to implicitly supervise retrieval, avoiding reliance on weakly aligned or parallel data.

Experimental results

Research questions

• RQ1 Can a grounded generation model be effectively trained without parallel retrieval-generation data?
• RQ2 How does joint training of retrieval and generation improve factual consistency and relevance in generated text?
• RQ3 To what extent can a language model signal alone guide effective document retrieval for generation?
• RQ4 Does the Mixture-of-Experts architecture enhance the integration of retrieved documents in generation?
• RQ5 Can the joint framework generalize across diverse generation tasks such as prose and dialogue?

Key findings

• The joint training framework significantly improves factual consistency and relevance in generated text compared to baseline models.
• The retriever learns to identify documents with high utility for generation without requiring explicit supervision or parallel data.
• The generator benefits from the retrieved documents, producing more informative and contextually accurate outputs.
• The Mixture-of-Experts mechanism enables effective and dynamic integration of multiple retrieved documents during generation.
• The model achieves strong performance in both open-ended text generation and dialogue tasks, demonstrating generalization across domains.
• The framework reduces hallucinations by grounding generation in retrieved evidence, even without parallel training data.