[Paper Review] Finetuned Language Models Are Zero-Shot Learners

Jason Wei, Maarten Bosma+7 more|arXiv (Cornell University)|Sep 3, 2021

Topic Modeling119 references69 citations

TL;DR

Instruction tuning a 137B language model on 60+ datasets described via natural language instructions substantially improves zero-shot performance on unseen tasks, surpassing GPT-3 zero-shot on many datasets and beating GPT-3 few-shot on several benchmarks.

ABSTRACT

This paper explores a simple method for improving the zero-shot learning abilities of language models. We show that instruction tuning -- finetuning language models on a collection of tasks described via instructions -- substantially improves zero-shot performance on unseen tasks. We take a 137B parameter pretrained language model and instruction-tune it on over 60 NLP tasks verbalized via natural language instruction templates. We evaluate this instruction-tuned model, which we call FLAN, on unseen task types. FLAN substantially improves the performance of its unmodified counterpart and surpasses zero-shot 175B GPT-3 on 20 of 25 tasks that we evaluate. FLAN even outperforms few-shot GPT-3 by a large margin on ANLI, RTE, BoolQ, AI2-ARC, OpenbookQA, and StoryCloze. Ablation studies reveal that number of finetuning datasets, model scale, and natural language instructions are key to the success of instruction tuning.

Motivation & Objective

• Demonstrate that finetuning a language model on tasks described with natural language instructions improves zero-shot task performance.
• Show how instruction tuning with a large, diverse dataset mixture affects generalization to unseen task types.
• Assess the role of model scale, number of training tasks, and instruction quality in zero-shot gains.

Proposed method

• Fine-tune a 137B parameter decoder-only language model (LaMDA-PT) on a mixture of over 60 NLP datasets expressed through natural language instruction templates.
• Create ten instruction templates per dataset to describe the task, including some templates that invert the task to diversify prompts.
• Evaluate zero-shot performance on held-out task clusters not seen during instruction tuning, using cluster-based evaluation splits.
• Use an options suffix to improve classification task outputs by specifying allowed choices at inference.
• Compare zero-shot and few-shot GPT-3, GPT-3 175B, and GLaM across evaluated datasets.
• Conduct ablations on the number of instruction-tuning clusters, model scale, role of instructions, and the impact of few-shot exemplars.

Experimental results

Research questions

• RQ1 Does finetuning a language model on instruction-formatted tasks improve its zero-shot performance on unseen task types?
• RQ2 How do factors like the number of instruction-tuning tasks, model scale, and instruction content influence zero-shot generalization?
• RQ3 To what extent can instruction-tuned models outperform GPT-3 in zero-shot and few-shot settings across diverse NLP tasks?
• RQ4 Are instructions alone responsible for gains, or do dataset templates and task clustering play crucial roles?
• RQ5 What is the interaction between instruction tuning and other prompting methods (few-shot prompts, prompt tuning)?

Key findings

• Instruction tuning substantially improves zero-shot performance on unseen tasks compared to the untuned base model.
• FLAN outperforms zero-shot GPT-3 on 20 of 25 evaluated datasets and can surpass GPT-3 few-shot performance on several tasks.
• The benefits of instruction tuning increase with more task clusters and larger model scale, with diminished gains for smaller models.
• Instruction tuning does not uniformly improve all tasks; gains are strongest on instruction-verbalizable tasks (NLI, QA, translation) and less for tasks framed as language modeling like some commonsense/coreference tasks.
• Few-shot exemplars further improve FLAN performance and reduce template sensitivity, especially for tasks with large output spaces.
• Instruction tuning also enhances compatibility with prompt tuning, offering improvements in various regimes.