[Paper Review] Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

Haokun Liu, Derek Tam+5 more|arXiv (Cornell University)|May 11, 2022

Topic Modeling292 citations

TL;DR

The paper shows that few-shot parameter-efficient fine-tuning (PEFT), specifically a new IA3-based method with a T-Few recipe, outperforms in-context learning (ICL) in accuracy while dramatically reducing computation, memory, and storage costs; it also achieves super-human performance on RAFT.

ABSTRACT

Few-shot in-context learning (ICL) enables pre-trained language models to perform a previously-unseen task without any gradient-based training by feeding a small number of training examples as part of the input. ICL incurs substantial computational, memory, and storage costs because it involves processing all of the training examples every time a prediction is made. Parameter-efficient fine-tuning (PEFT) (e.g. adapter modules, prompt tuning, sparse update methods, etc.) offers an alternative paradigm where a small set of parameters are trained to enable a model to perform the new task. In this paper, we rigorously compare few-shot ICL and PEFT and demonstrate that the latter offers better accuracy as well as dramatically lower computational costs. Along the way, we introduce a new PEFT method called (IA)$^3$ that scales activations by learned vectors, attaining stronger performance while only introducing a relatively tiny amount of new parameters. We also propose a simple recipe based on the T0 model called T-Few that can be applied to new tasks without task-specific tuning or modifications. We validate the effectiveness of T-Few on completely unseen tasks by applying it to the RAFT benchmark, attaining super-human performance for the first time and outperforming the state-of-the-art by 6% absolute. All of the code used in our experiments is publicly available.

Motivation & Objective

• Motivate a data-efficient alternative to ICL for adapting pre-trained language models to new tasks with very limited labeled data.
• Propose a concrete PEFT-based recipe that scales to mixed-task inference and requires minimal parameter updates.
• Enhance few-shot performance with additional loss terms to better align training objectives with ranking-based evaluation.

Proposed method

• Use T0-3B as the base model and apply a new PEFT method (IA)3 that multiplicatively modulates selected activations.
• Introduce three learned vectors (l_k, l_v, l_ff) to rescale keys, values, and feed-forward activations in a layer-wise manner.
• Train with a combined objective L = L_LM + L_UL + L_LN to improve ranking of correct choices while discouraging incorrect ones.
• Pre-train the IA3 parameters on the multitask mixture used for T0, then fine-tune on downstream tasks.
• Apply fixed prompting templates (P3) and train for 1,000 steps with Adafactor, using a uniform recipe across tasks without per-task hyperparameter tuning.

Experimental results

Research questions

• RQ1 Can PEFT with a lightweight, activations-based adaptation (IA3) match or exceed full-model fine-tuning in few-shot settings?
• RQ2 Does the T-Few recipe generalize across unseen tasks, enabling mixed-task inference with small labeled datasets?
• RQ3 How do additional loss terms (unlikelihood and length normalization) affect few-shot learning performance?
• RQ4 What are the comparative compute, memory, and storage costs of T-Few versus in-context learning across tasks?
• RQ5 Is the approach effective on real-world few-shot benchmarks like RAFT?

Key findings

• T-Few consistently outperforms strong ICL baselines, including GPT-3 variants, on held-out T0 tasks.
• T-Few achieves 72.4% accuracy on held-out tasks versus GPT-3 175B at 66.6% and T0 at 66.9%.
• On RAFT, T-Few reaches 75.8% accuracy, surpassing the human baseline of 73.5%.
• IA3 is the only PEFT method among those tested that surpasses full-model fine-tuning in accuracy while updating far fewer parameters (up to 10,000× fewer).
• Training T-Few (IA3) adds about 4.2 MB on disk, while inference costs are dramatically lower than ICL (GPT-3 175B costs exceed 1e15 FLOPs for 41-shot contexts).
• Pre-training IA3 parameters before task-specific fine-tuning improves accuracy (from 64.6 to 65.8).