[Paper Review] Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation

M. Boubdir, Edward Kim+3 more|arXiv (Cornell University)|Oct 22, 2023

Topic Modeling4 citations

TL;DR

This paper proposes a data prioritization strategy for efficient human evaluation of large language models by ranking prompts based on KL divergence and cross-entropy to maximize model discrimination. It reduces tie rates by up to 54% in the top 20% of prioritized instances, significantly cutting human annotation effort while maintaining robust performance rankings via stable Elo scores.

ABSTRACT

Human evaluation is increasingly critical for assessing large language models, capturing linguistic nuances, and reflecting user preferences more accurately than traditional automated metrics. However, the resource-intensive nature of this type of annotation process poses significant challenges. The key question driving our work: "is it feasible to minimize human-in-the-loop feedback by prioritizing data instances which most effectively distinguish between models?" We evaluate several metric-based methods and find that these metrics enhance the efficiency of human evaluations by minimizing the number of required annotations, thus saving time and cost, while ensuring a robust performance evaluation. We show that our method is effective across widely used model families, reducing instances of indecisive (or "tie") outcomes by up to 54% compared to a random sample when focusing on the top-20 percentile of prioritized instances. This potential reduction in required human effort positions our approach as a valuable strategy in future large language model evaluations.

Motivation & Objective

• To reduce the resource burden of human-in-the-loop LLM evaluation by minimizing required annotations.
• To identify which prompts most effectively distinguish between model responses in pairwise comparisons.
• To mitigate indecisive (tie) outcomes in human evaluations, which waste time and reduce evaluation efficiency.
• To improve the robustness and stability of model ranking using Elo scores through smarter prompt selection.
• To develop a systematic, offline method for prompt prioritization that scales across diverse model families.

Proposed method

• Rank prompts using KL divergence and cross-entropy between model completions to estimate dissimilarity.
• Prioritize the top-performing prompts—especially those with high dissimilarity—for human evaluation.
• Use pairwise comparison format where human evaluators rank model responses on a prompt-by-prompt basis.
• Apply Elo scoring to model rankings and track convergence and stability across evaluation sequences.
• Benchmark the method across multiple model families including Flan-T5, Dolly-V2, MPT, and Falcon.
• Compare performance against random sampling to isolate gains from prioritization.

Experimental results

Research questions

• RQ1 Which prompts yield the most decisive human judgments in pairwise model comparisons?
• RQ2 Can metric-based prompt prioritization reduce the number of required human annotations without sacrificing evaluation reliability?
• RQ3 To what extent does prompt prioritization reduce tie outcomes in human evaluations?
• RQ4 How does prompt ranking affect the stability and convergence of Elo-based model rankings?
• RQ5 Does the method generalize across diverse model families with different architectures and training data?

Key findings

• The method reduces tie rates by up to 54% in the top 20% of prioritized prompts compared to random sampling.
• KL divergence-based ranking outperforms cross-entropy and random selection in minimizing indecisive outcomes.
• Elo scores converge faster and show greater stability when using prioritized prompts, especially in early evaluation stages.
• The approach maintains robust performance evaluation across diverse model families, including Flan-T5, Dolly-V2, MPT, and Falcon.
• Intra-family model comparisons (e.g., Dolly-V2-7B vs. 12B) show higher tie rates, which are effectively reduced through prioritization.
• The method enables reliable model ranking with significantly fewer human annotations, reducing time and cost.