[Paper Review] Prefix-Tuning: Optimizing Continuous Prompts for Generation

Xiang Lisa Li, Percy Liang|arXiv (Cornell University)|Jan 1, 2021

Topic Modeling40 references290 citations

TL;DR

Prefix-tuning freezes the language model and trains a small continuous prefix to steer generation, achieving comparable results to fine-tuning with 1000x fewer parameters and better performance in low-data and extrapolation settings.

ABSTRACT

Fine-tuning is the de facto way to leverage large pretrained language models to perform downstream tasks. However, it modifies all the language model parameters and therefore necessitates storing a full copy for each task. In this paper, we propose prefix-tuning, a lightweight alternative to fine-tuning for natural language generation tasks, which keeps language model parameters frozen, but optimizes a small continuous task-specific vector (called the prefix). Prefix-tuning draws inspiration from prompting, allowing subsequent tokens to attend to this prefix as if it were "virtual tokens". We apply prefix-tuning to GPT-2 for table-to-text generation and to BART for summarization. We find that by learning only 0.1\% of the parameters, prefix-tuning obtains comparable performance in the full data setting, outperforms fine-tuning in low-data settings, and extrapolates better to examples with topics unseen during training.

Motivation & Objective

• Motivate a lightweight alternative to full fine-tuning for large pretrained LMs in generation tasks.
• Introduce a method to train a small continuous prefix while keeping LM parameters fixed.
• Demonstrate efficiency and effectiveness of prefix-tuning on table-to-text and abstractive summarization.
• Evaluate prefix-tuning in full-data, low-data, and extrapolation scenarios to assess generalization and scalability.

Proposed method

• Prepend a trainable continuous prefix to inputs for autoregressive LMs or encoder-decoder models.
• Represent the prefix with a trainable matrix P_theta of size |P_idx| x dim(h_i), reparameterized via an MLP for stability.
• Freeze LM parameters phi and optimize only the prefix parameters theta during training.
• Initialize prefix activations as either random or with real-word activations, showing initialization impact especially in low-data settings.
• Compare prefix-tuning to fine-tuning, top-layer fine-tuning, and adapter-tuning across table-to-text (GPT-2 variants) and summarization (BART).
• Use standard generation objectives (log-likelihood) with consistent decoding setups (beam search, etc.).

Experimental results

Research questions

• RQ1 Can a small continuous prefix steer generation tasks without updating the full LM parameters?
• RQ2 How does prefix-tuning compare to full fine-tuning and lightweight baselines in table-to-text and summarization under full-data and low-data regimes?
• RQ3 Does prefix-tuning generalize to unseen topics or domains (extrapolation) better than fine-tuning?
• RQ4 What is the effect of prefix length, initialization, and position (prefix vs infix) on performance?
• RQ5 Is prefix-tuning scalable and batch-friendly for personalization scenarios with many tasks/users?

Key findings

• With 0.1% of parameters updated, prefix-tuning achieves comparable performance to fine-tuning on table-to-text.
• Prefix-tuning outperforms fine-tuning in low-data settings across tasks.
• Prefix-tuning demonstrates better extrapolation to unseen topics compared to fine-tuning.
• Prefix-tuning scales from GPT-2 medium to large, maintaining strong performance while drastically reducing task-specific parameters.
• Initialization with activations of real words improves performance, especially in low-data regimes.
• Prefix-tuning remains Pareto-efficient relative to adapter-tuning, achieving similar or better results with far fewer parameters.