[Paper Review] LaMDA: Language Models for Dialog Applications

Romal Thoppilan, Daniel De Freitas+58 more|arXiv (Cornell University)|Jan 20, 2022

Topic ModelingComputer Science705 citations

TL;DR

LaMDA is a family of large, Transformer-based dialog models (up to 137B parameters) pre-trained on public dialog and web data, and refined with fine-tuning and external tools to improve dialog quality, safety, and grounding.

ABSTRACT

We present LaMDA: Language Models for Dialog Applications. LaMDA is a family of Transformer-based neural language models specialized for dialog, which have up to 137B parameters and are pre-trained on 1.56T words of public dialog data and web text. While model scaling alone can improve quality, it shows less improvements on safety and factual grounding. We demonstrate that fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements towards the two key challenges of safety and factual grounding. The first challenge, safety, involves ensuring that the model's responses are consistent with a set of human values, such as preventing harmful suggestions and unfair bias. We quantify safety using a metric based on an illustrative set of human values, and we find that filtering candidate responses using a LaMDA classifier fine-tuned with a small amount of crowdworker-annotated data offers a promising approach to improving model safety. The second challenge, factual grounding, involves enabling the model to consult external knowledge sources, such as an information retrieval system, a language translator, and a calculator. We quantify factuality using a groundedness metric, and we find that our approach enables the model to generate responses grounded in known sources, rather than responses that merely sound plausible. Finally, we explore the use of LaMDA in the domains of education and content recommendations, and analyze their helpfulness and role consistency.

Motivation & Objective

• Investigate how model scaling affects dialog quality, safety, and groundedness.
• Develop fine-tuning strategies using annotated dialog data to improve safety and response quality.
• Enable external knowledge access through tools to enhance grounding and factual accuracy.
• Evaluate LaMDA in education and content recommendation scenarios to assess helpfulness and role consistency.

Proposed method

• Train decoder-only Transformer models up to 137B parameters on 1.56T words of public dialog and web data.
• Use a sample-and-rank strategy to generate and select candidate responses based on log-likelihood and length.
• Fine-tune with discriminative and generative objectives to optimize quality (SSI) and safety.
• Augment outputs with an external toolset (information retrieval, calculator, translator) and train the model to issue tool-use queries and incorporate retrieved snippets.
• Collect and use large-scale annotated datasets (dialog turns, safety annotations, grounding annotations) for evaluation and fine-tuning.
• Precondition LaMDA on application-specific dialog to assess role-specific helpfulness and consistency.

Experimental results

Research questions

• RQ1 How does scaling (model size) impact dialog quality, safety, and groundedness compared to human performance?
• RQ2 Does fine-tuning combined with scaling improve quality, safety, and grounding beyond scaling alone?
• RQ3 Can grounding be improved by enabling the model to consult external knowledge sources and tools?
• RQ4 How does LaMDA perform in education and content recommendation domains in terms of helpfulness and role consistency?

Key findings

• Model scaling improves dialog quality (sensibleness, specificity, interestingness).
• Scaling alone yields limited improvements in safety and groundedness relative to human performance.
• Fine-tuning plus scaling yields significant gains across quality, safety, and groundedness.
• Augmenting outputs with an external knowledge toolset improves groundedness and reduces ungrounded claims.
• Discriminative fine-tuning and a separate safety predictor help filter unsafe responses before ranking candidates.
• LaMDA's application-specific fine-tuned variants are more helpful and maintain role consistency in education and content recommendation scenarios.