[Paper Review] Calibrate Before Use: Improving Few-Shot Performance of Language Models

Tony Z. Zhao, Eric Wallace+3 more|arXiv (Cornell University)|Feb 19, 2021

Topic ModelingComputer Science33 references72 citations

TL;DR

The paper introduces contextual calibration to fix prompt-induced biases in few-shot learning with GPT-3/GPT-2, significantly boosting accuracy and reducing prompt variance without extra training data.

ABSTRACT

GPT-3 can perform numerous tasks when provided a natural language prompt that contains a few training examples. We show that this type of few-shot learning can be unstable: the choice of prompt format, training examples, and even the order of the training examples can cause accuracy to vary from near chance to near state-of-the-art. We demonstrate that this instability arises from the bias of language models towards predicting certain answers, e.g., those that are placed near the end of the prompt or are common in the pre-training data. To mitigate this, we first estimate the model's bias towards each answer by asking for its prediction when given the training prompt and a content-free test input such as "N/A". We then fit calibration parameters that cause the prediction for this input to be uniform across answers. On a diverse set of tasks, this contextual calibration procedure substantially improves GPT-3 and GPT-2's average accuracy (up to 30.0% absolute) and reduces variance across different choices of the prompt.

Motivation & Objective

• Motivate and quantify instability in few-shot learning with in-context prompts for large language models.
• Identify biases (majority label, recency, common token) that drive unstable predictions.
• Propose a data-free calibration method to align model outputs across answers.
• Demonstrate calibration improves accuracy and reduces variance across diverse tasks and prompt formats.

Proposed method

• Identify three source of bias causing instability in few-shot prompts: majority label bias, recency bias, and common token bias.
• Propose a data-free calibration procedure that uses a content-free input (e.g., 'N/A') to estimate per-answer bias.
• Compute a diagonal affine calibration (vector scaling) to transform output probabilities so the content-free case is uniform across answers.
• Apply calibration to GPT-3, GPT-2 (and GPT-3 variants) across multiple tasks with fixed prompts and varied formats/examples.
• Demonstrate that calibration can be implemented with minimal overhead and no additional training data.

Experimental results

Research questions

• RQ1 How unstable is few-shot performance across prompt formats, example selections, and orderings for large language models?
• RQ2 What biases drive the variability in model predictions during in-context learning?
• RQ3 Can a data-free calibration procedure align outputs to be more task-appropriate and robust to prompt variations?
• RQ4 To what extent does contextual calibration improve accuracy and reduce variance across diverse NLP tasks?

Key findings

• Contextual calibration yields up to 30.0% absolute improvement in mean accuracy across tasks and prompts.
• Calibration reduces performance variance caused by different prompt choices and example permutations.
• GPT-3 2.7B can outperform the GPT-3 175B baseline on some tasks after calibration (up to 19.3% absolute).
• Calibration improves GPT-2 similarly, indicating the method generalizes beyond GPT-3.
• The approach works across text classification, fact retrieval (LAMA), and information extraction, demonstrating broad applicability.