[Paper Review] Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge

Lin Shi, Weicheng Ma+4 more|arXiv (Cornell University)|Jun 12, 2024

Legal Education and Practice Innovations4 citations

TL;DR

This paper systematically investigates position bias in LLM-as-a-Judge systems, where models favor responses based on their order in pairwise comparisons rather than quality. Using a framework with metrics like positional consistency and fairness across 9 judge models on 22 tasks, it finds significant bias variation—GPT-4 shows strong consistency and fairness, but cheaper models like GPT-3.5 perform comparably in some tasks, highlighting trade-offs between cost, consistency, and fairness.

ABSTRACT

LLM-as-a-Judge has emerged as a promising alternative to human evaluators across various tasks, yet inherent biases - particularly position bias, the tendency to favor solutions based on their position within the prompt - compromise its reliability. This exploratory study evaluates position bias in LLM judges across pairwise and list-wise comparison settings, introducing three metrics: repetition stability, position consistency, and preference fairness. Our experiments, involving 15 LLM judges across MTBench and DevBench with 22 tasks and approximately 40 solution-generating models, result in over 150,000 evaluation instances. We identify Judge-Level, Candidate-Level, and Task-Level factors contributing to bias. The findings confirm that position bias is not due to random chance and varies significantly across judges and tasks. While position bias is weakly influenced by the length of prompt components, it is strongly affected by the quality gap between solutions. Our agreement and disagreement analysis among judges further provides insights into the distribution of judging difficulty across the dataset, and highlights the potential for dataset modifications.

Motivation & Objective

• To investigate and quantify position bias in LLM-as-a-Judge systems, where model judgments systematically favor responses based on their position in the prompt.
• To develop a standardized framework with metrics such as repetitional consistency, positional consistency, and positional fairness to evaluate and compare bias across judge models.
• To analyze the impact of model family, task type, and answer quality gap on position bias, and to assess whether length or verbosity biases drive preference.
• To provide actionable insights for selecting optimal judge models and improving benchmark design, ultimately enhancing the reliability of automated LLM evaluation.
• To establish foundational terminology and metrics for future research into debiasing strategies in LLM-based evaluation systems.

Proposed method

• Designed a systematic evaluation framework to measure position bias using pairwise comparative assessments, where two responses are evaluated in both original and swapped orders.
• Defined key metrics: repetitional consistency (judgment stability across repeated evaluations), positional consistency (preference for first or second response), and positional fairness (balanced preference across positions).
• Operationalized terms like positional preference score, primacy/recency preference, and answer quality gap to standardize bias analysis across models and tasks.
• Conducted experiments across 9 LLM judges (GPT-3.5, GPT-4, GPT-4-Turbo, Claude-3, Gemini-Pro) on 22 tasks from MTBench and DevBench, generating ~80,000 evaluation instances.
• Applied OLS regression to analyze the influence of task-level factors, including input length, output length, and prompt length, on positional consistency and preference.
• Visualized and analyzed relationships between length statistics and bias, concluding that length effects are minimal and that observed verbosity bias likely stems from position bias and quality gaps.

Experimental results

Research questions

• RQ1 To what extent do LLM judges exhibit position bias in pairwise comparative assessments, and how does this vary across different model families and tasks?
• RQ2 How do metrics like positional consistency, fairness, and repetitional consistency correlate with model performance and intrinsic properties such as context window and architecture?
• RQ3 Does response length or verbosity independently drive preference, or is the observed bias better explained by positional ordering and answer quality gap?
• RQ4 How does the answer quality gap influence the consistency and fairness of LLM judgments, particularly in cases with closely matched responses?
• RQ5 Can cost-effective LLMs achieve comparable or better fairness and consistency than high-cost models like GPT-4 in specific evaluation tasks?

Key findings

• Position bias is systematic and reproducible: 80% of judgments showed consistent preferences across repetitions, indicating it is not due to randomness but to model-specific structural traits.
• GPT-4 demonstrated the highest positional consistency (0.92) and fairness (0.51), though some cheaper models like GPT-3.5 showed comparable or better fairness in specific tasks.
• Familial model traits significantly influence bias: GPT-series models showed strong consistency and fairness, while Claude-3 models displayed a notable recency preference (higher preference for second responses).
• Positional consistency was directly proportional to the answer quality gap—judgments were less consistent when responses were of similar quality, highlighting challenges in fine-grained evaluation.
• Length-based biases (e.g., favoring longer responses) were not independently significant; instead, they were subsumed under position bias, suggesting that length preference is a proxy for quality bias.
• OLS regression showed that task-specific factors (e.g., UML class design) had a statistically significant impact on positional preference (p < 0.05), while model family effects like GPT-4 Turbo showed negative bias (β = -0.0036, p = 0.015), indicating a slight preference for the second response.