[Paper Review] Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity

Yao Lu, Max Bartolo+3 more|arXiv (Cornell University)|Apr 18, 2021

Topic Modeling119 citations

TL;DR

The paper shows that prompt sample order in in-context learning dramatically affects performance across models and tasks, and introduces entropy-based probing to automatically identify performant orders without extra labeled data, yielding notable improvements.

ABSTRACT

When primed with only a handful of training samples, very large, pretrained language models such as GPT-3 have shown competitive results when compared to fully-supervised, fine-tuned, large, pretrained language models. We demonstrate that the order in which the samples are provided can make the difference between near state-of-the-art and random guess performance: essentially some permutations are "fantastic" and some not. We analyse this phenomenon in detail, establishing that: it is present across model sizes (even for the largest current models), it is not related to a specific subset of samples, and that a given good permutation for one model is not transferable to another. While one could use a development set to determine which permutations are performant, this would deviate from the true few-shot setting as it requires additional annotated data. Instead, we use the generative nature of language models to construct an artificial development set and based on entropy statistics of the candidate permutations on this set, we identify performant prompts. Our method yields a 13% relative improvement for GPT-family models across eleven different established text classification tasks.

Motivation & Objective

• Demonstrate that sample order in in-context learning significantly impacts performance across model sizes and tasks.
• Show that good permutations are not transferable across models or tasks.
• Propose a probing-based method to automatically identify performant prompt orders without labeled development data.
• Leverage the generative nature of LMs to construct an unlabeled probing set for evaluation.
• Quantify the improvement gained by entropy-based probing across diverse datasets and models.

Proposed method

• Analyze order sensitivity across GPT-2/GPT-3 models using 4-shot prompts on SST-2 and other datasets.
• Construct a probing set by sampling from the language model to generate unlabeled examples corresponding to training samples.
• Define Global Entropy and Local Entropy metrics to rank candidate prompt orders based on probing set predictions.
• Select top-k prompt orders (k=4) with highest entropy scores and evaluate on multiple datasets.
• Demonstrate that entropy-based probing improves average performance by ~13% relative across eleven tasks.

Experimental results

Research questions

• RQ1 Is few-shot prompt order sensitivity universal across model sizes and tasks?
• RQ2 Can we identify performant prompt orders automatically without labeled development data?
• RQ3 Do good prompt permutations transfer between models or tasks?
• RQ4 Are entropy-based probing metrics robust to different templates and datasets?

Key findings

• Prompt order variability can cause performance to swing from near state-of-the-art to near random across GPT-2/GPT-3 sizes.
• A single good permutation does not reliably transfer to different models or datasets.
• Global Entropy and Local Entropy reliably identify performant prompt orders using an unlabeled probing set.
• Entropy-based probing yields an average 13% relative improvement across eleven text classification tasks (across model sizes).
• Selected prompts show substantially reduced performance variance compared with using all candidate orders.
• Entropy-based probing outperforms simple validation-set tuning or data-splitting methods for prompt selection.