[Paper Review] ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs

Jingming Zhuo, Songyang Zhang+4 more|arXiv (Cornell University)|Oct 16, 2024

Biomedical Text Mining and Ontologies8 citations

TL;DR

ProSA introduces PromptSensiScore (PSS) to measure instance-level prompt sensitivity in LLMs, analyzes objective and subjective evaluation robustness, and links sensitivity to decoding confidence.

ABSTRACT

Large language models (LLMs) have demonstrated impressive capabilities across various tasks, but their performance is highly sensitive to the prompts utilized. This variability poses challenges for accurate assessment and user satisfaction. Current research frequently overlooks instance-level prompt variations and their implications on subjective evaluations. To address these shortcomings, we introduce ProSA, a framework designed to evaluate and comprehend prompt sensitivity in LLMs. ProSA incorporates a novel sensitivity metric, PromptSensiScore, and leverages decoding confidence to elucidate underlying mechanisms. Our extensive study, spanning multiple tasks, uncovers that prompt sensitivity fluctuates across datasets and models, with larger models exhibiting enhanced robustness. We observe that few-shot examples can alleviate this sensitivity issue, and subjective evaluations are also susceptible to prompt sensitivities, particularly in complex, reasoning-oriented tasks. Furthermore, our findings indicate that higher model confidence correlates with increased prompt robustness. We believe this work will serve as a helpful tool in studying prompt sensitivity of LLMs. The project is released at: https://github.com/open-compass/ProSA .

Motivation & Objective

• Motivate and quantify how prompts influence LLM responses at the instance level rather than dataset-level.
• Develop an instance-level sensitivity metric (PSS) for objective and subjective evaluations.
• Investigate how prompt sensitivity varies across models, datasets, and prompt styles.
• Explore the relationship between decoding confidence and prompt robustness.

Proposed method

• Define PSS as the average pairwise discrepancy in responses across all prompt variants for the same instance.
• Evaluate PSS across multiple open-source LLMs and datasets in objective tasks.
• Assess the impact of few-shot prompts on prompt robustness.
• Perform subjective evaluation analysis using LC AlpacaEval 2.0 and Arena Hard Auto with prompt rewriting.
• Use decoding confidence to analyze underlying causes of prompt sensitivity.

Figure 1: A Showcase of the Four Prompt Templates on MATH. These four prompt templates represent four different styles of constructing prompts, serving as an example of the diversity in human prompt expression.

Experimental results

Research questions

• RQ1 How does instance-level prompt sensitivity vary across datasets and model sizes?
• RQ2 Does few-shot prompting reduce prompt sensitivity and increase robustness?
• RQ3 What is the relationship between an LLM’s decoding confidence and its prompt robustness in practice?
• RQ4 How does prompt sensitivity manifest in subjective evaluations compared to objective evaluations?
• RQ5 Which task categories show higher or lower prompt sensitivity?

Key findings

• Prompt sensitivity varies by dataset and model, with larger models often showing greater robustness.
• Few-shot prompting reduces prompt sensitivity, especially when moving from 0-shot to 1-shot settings, and larger models gain robustness with more shots.
• Subjective evaluations reveal higher sensitivity on complex tasks, despite robustness on simpler ones.
• Decoding confidence correlates with lower prompt sensitivity, suggesting robustness reflects underlying decoding dynamics.
• Prompts categories influence sensitivity; knowledge-heavy tasks tend to be more robust than coding or creative tasks.

Figure 2: A Comparision of Evaluating LLMs’ Prompt Sensitivity. ✓ and ✗ indicate the accuracy of the LLM’s responses. In this example, LLMs appear robust at the dataset level evaluation (calculated from the variance of different templates), but this overlooks the sensitivity of LLMs to different tem