[Paper Review] Investigating Implicit Bias in Large Language Models: A Large-Scale Study of Over 50 LLMs

Divyanshu Kumar, Umang Jain+2 more|arXiv (Cornell University)|Oct 13, 2024

Natural Language Processing Techniques4 citations

TL;DR

This large-scale study investigates implicit bias in over 50 large language models using the LLM Implicit Association Test (IAT) Bias and LLM Decision Bias frameworks. It reveals that newer or larger models often exhibit higher implicit bias than their predecessors, suggesting that increased model size without deliberate mitigation can amplify existing biases, highlighting the urgent need for standardized bias evaluation and proactive fairness measures in LLM development.

ABSTRACT

Large Language Models (LLMs) are being adopted across a wide range of tasks, including decision-making processes in industries where bias in AI systems is a significant concern. Recent research indicates that LLMs can harbor implicit biases even when they pass explicit bias evaluations. Building upon the frameworks of the LLM Implicit Association Test (IAT) Bias and LLM Decision Bias, this study highlights that newer or larger language models do not automatically exhibit reduced bias; in some cases, they displayed higher bias scores than their predecessors, such as in Meta's Llama series and OpenAI's GPT models. This suggests that increasing model complexity without deliberate bias mitigation strategies can unintentionally amplify existing biases. The variability in bias scores within and across providers underscores the need for standardized evaluation metrics and benchmarks for bias assessment. The lack of consistency indicates that bias mitigation is not yet a universally prioritized goal in model development, which can lead to unfair or discriminatory outcomes. By broadening the detection of implicit bias, this research provides a more comprehensive understanding of the biases present in advanced models and underscores the critical importance of addressing these issues to ensure the development of fair and responsible AI systems.

Motivation & Objective

• To investigate the presence and evolution of implicit bias in large language models across model size and age.
• To assess whether larger or newer LLMs inherently exhibit reduced bias, challenging the assumption that scale improves fairness.
• To evaluate the effectiveness of LLM IAT Bias and LLM Decision Bias as detection frameworks for implicit bias in proprietary, black-box models.
• To identify patterns in bias amplification, particularly linked to synthetic data usage in newer model training.
• To advocate for standardized, transparent bias evaluation benchmarks to ensure fairness in LLM development and deployment.

Proposed method

• Applied the LLM Implicit Association Test (IAT) Bias framework to measure implicit associations between demographic attributes and valence terms (e.g., 'wonderful' vs. 'awful').
• Utilized the LLM Decision Bias metric to evaluate model preferences between stereotypical and anti-stereotypical sentence pairs.
• Conducted large-scale inference on 50+ LLMs, including models from Meta (Llama), OpenAI (GPT), and others, using prompt-based evaluation without requiring model weights.
• Collected and analyzed model responses to standardized prompts involving race, gender, and socioeconomic context to infer bias patterns.
• Compared bias scores across model families, sizes, and release dates to identify trends in bias evolution.
• Used synthetic data exposure as a hypothesized driver of increased bias in newer models, based on observed correlations in model performance and training data patterns.

Figure 1: Bias Evaluation Pipeline

Experimental results

Research questions

• RQ1 Do larger or newer LLMs exhibit lower implicit bias, or does model scale amplify existing biases?
• RQ2 How consistent are implicit bias scores across different LLM providers and model architectures?
• RQ3 To what extent do synthetic training data and model fine-tuning contribute to increased implicit bias in newer models?
• RQ4 Can prompt-based evaluation frameworks like LLM IAT and Decision Bias reliably detect implicit bias in black-box LLMs?
• RQ5 What are the implications of inconsistent bias scores for fairness and accountability in real-world LLM deployment?

Key findings

• Newer and larger LLMs, including Meta’s Llama series and OpenAI’s GPT models, often exhibit higher implicit bias scores than their predecessors, contradicting the assumption that scale reduces bias.
• The study found no consistent reduction in bias with model size or release date, indicating that implicit bias is not automatically mitigated through scaling.
• Bias scores varied significantly across models from the same provider, suggesting a lack of standardized bias mitigation in model development pipelines.
• The researchers hypothesize that increased use of synthetic data in training newer models may contribute to higher implicit bias, though this requires further validation.
• The LLM IAT and Decision Bias frameworks proved effective in detecting implicit bias in proprietary, API-accessible LLMs, enabling large-scale black-box evaluation.
• The findings underscore the critical need for standardized, transparent, and mandatory bias evaluation benchmarks in LLM development to ensure fairness and prevent discriminatory outcomes.