[Paper Review] Prompt Consistency for Zero-Shot Task Generalization

Chunting Zhou, Junxian He+3 more|arXiv (Cornell University)|Apr 29, 2022

Topic Modeling4 citations

TL;DR

This paper proposes prompt consistency regularization via swarm distillation to improve zero-shot task generalization in large language models using unlabeled data. By encouraging consistent predictions across multiple synonymous prompts for the same input, the method achieves state-of-the-art performance on 9 out of 11 NLP benchmarks, improving accuracy by up to 10.6 points with minimal labeled data and only a few unlabeled examples.

ABSTRACT

One of the most impressive results of recent NLP history is the ability of pre-trained language models to solve new tasks in a zero-shot setting. To achieve this, NLP tasks are framed as natural language prompts, generating a response indicating the predicted output. Nonetheless, the performance in such settings often lags far behind its supervised counterpart, suggesting a large space for potential improvement. In this paper, we explore methods to utilize unlabeled data to improve zero-shot performance. Specifically, we take advantage of the fact that multiple prompts can be used to specify a single task, and propose to regularize prompt consistency, encouraging consistent predictions over this diverse set of prompts. Our method makes it possible to fine-tune the model either with extra unlabeled training data, or directly on test input at inference time in an unsupervised manner. In experiments, our approach outperforms the state-of-the-art zero-shot learner, T0 (Sanh et al., 2022), on 9 out of 11 datasets across 4 NLP tasks by up to 10.6 absolute points in terms of accuracy. The gains are often attained with a small number of unlabeled examples.

Motivation & Objective

• To improve zero-shot generalization of pre-trained language models without relying on labeled data.
• To address the sensitivity of model predictions to prompt wording, which limits zero-shot performance.
• To develop an unsupervised method that leverages unlabeled data to enhance robustness across diverse prompts for the same task.
• To mitigate catastrophic forgetting and model collapse during unsupervised fine-tuning through parameter-efficient tuning and checkpoint selection.
• To demonstrate practicality and scalability of the method across diverse NLP tasks with minimal data and computation.

Proposed method

• The method introduces a pairwise distillation loss that regularizes predictions to be consistent across multiple synonymous prompts for the same input.
• It employs swarm distillation, a fully unsupervised training objective that uses unlabeled inputs to enforce consistency across diverse prompt variants.
• The approach uses parameter-efficient fine-tuning (e.g., LoRA) to update only a small subset of parameters, reducing risk of catastrophic forgetting.
• An unsupervised criterion based on Fleiss’ Kappa is used to select model checkpoints before collapse into a uniform prediction behavior.
• The method is applied both during training with unlabeled data and at inference time via test-time tuning on unlabeled test inputs.
• It is compatible with existing pre-trained models and can be combined with other objectives, such as masked language modeling or few-shot learning.

Experimental results

Research questions

• RQ1 Can prompt consistency regularization improve zero-shot generalization in pre-trained language models without any labeled data?
• RQ2 How effective is the proposed swarm distillation method in improving model robustness across diverse prompt variants?
• RQ3 What is the minimal amount of unlabeled data required to achieve significant performance gains?
• RQ4 Can the method be applied effectively at inference time using only test-time unlabeled examples?
• RQ5 How does the method compare to state-of-the-art zero-shot learners like T0 across diverse NLP tasks?

Key findings

• The proposed swarm distillation method outperforms the SOTA zero-shot learner T0 on 9 out of 11 datasets across 4 NLP tasks, with gains of up to 10.6 absolute points in accuracy.
• The method achieves significant improvements with as few as 10 unlabeled examples, indicating low data and computational overhead.
• Performance saturates quickly with increasing unlabeled data, suggesting the method is not data-hungry and scales efficiently.
• The method maintains strong performance even when fine-tuned directly on test inputs at inference time, enabling effective test-time adaptation.
• Scaling the model to 11B parameters further improves performance, with the method outperforming the 11B T0 model on all 4 evaluated datasets.
• The approach is robust to prompt variation and reduces sensitivity to prompt wording, demonstrating improved generalization and consistency.