[Paper Review] A Survey on Fairness in Large Language Models

Yingji Li, Mengnan Du+3 more|arXiv (Cornell University)|Aug 20, 2023

Ethics and Social Impacts of AI21 citations

TL;DR

This survey reviews fairness in medium-scale LLMs (pre-training/fine-tuning) and large-scale LLMs (prompting), covering evaluation metrics, debiasing methods, reasons for bias, and future directions.

ABSTRACT

Large Language Models (LLMs) have shown powerful performance and development prospects and are widely deployed in the real world. However, LLMs can capture social biases from unprocessed training data and propagate the biases to downstream tasks. Unfair LLM systems have undesirable social impacts and potential harms. In this paper, we provide a comprehensive review of related research on fairness in LLMs. Considering the influence of parameter magnitude and training paradigm on research strategy, we divide existing fairness research into oriented to medium-sized LLMs under pre-training and fine-tuning paradigms and oriented to large-sized LLMs under prompting paradigms. First, for medium-sized LLMs, we introduce evaluation metrics and debiasing methods from the perspectives of intrinsic bias and extrinsic bias, respectively. Then, for large-sized LLMs, we introduce recent fairness research, including fairness evaluation, reasons for bias, and debiasing methods. Finally, we discuss and provide insight on the challenges and future directions for the development of fairness in LLMs.

Motivation & Objective

• Explain how fairness is defined and evaluated in medium-scale LLMs and large-scale LLMs.
• Summarize intrinsic and extrinsic debiasing methods for medium-scale LLMs.
• Survey fairness evaluation, bias causes, and debiasing approaches for large-scale LLMs under prompting.

Proposed method

• Classify fairness metrics into intrinsic (embeddings) and extrinsic (downstream tasks) for medium-scale LLMs.
• Review intrinsic debiasing methods: pre-processing, in-processing, post-processing.
• Review extrinsic debiasing methods: data-centric and model-centric approaches.
• Discuss fairness evaluation methods, bias causes, and debiasing strategies for large-scale LLMs under prompting.
• Synthesize challenges and future directions for LLM fairness.

Experimental results

Research questions

• RQ1 What evaluation metrics capture intrinsic and extrinsic fairness in medium-scale LLMs?
• RQ2 What debiasing methods exist at pre-processing, in-processing, and post-processing stages for medium-scale LLMs?
• RQ3 How is fairness evaluated and debiased in large-scale LLMs under prompting paradigms?
• RQ4 What are the main challenges and future directions for fairness in LLMs?
• RQ5 What are the known reasons behind bias in large-scale LLMs and how can they be mitigated?

Key findings

• Intrinsic metrics quantify bias in embeddings (e.g., SEAT, CEAT, DisCo, LPBS, STS-B) while extrinsic metrics assess downstream task fairness (coreference, semantic similarity, group fairness).
• Medium-scale debiasing spans data-level, model-level, and output-level techniques across pre-processing, in-processing, and post-processing.
• Large-scale LLM fairness is evaluated via prompts, dialogues, and bias benchmarks (e.g., BBQ, BiasAsker), with evidence of persistent bias despite improvements in some settings.
• Bias in large-scale LLMs arises from training data biases, model frequency effects, and world-knowledge representations, with phase transitions in capabilities influencing bias patterns.
• Debiasing large-scale LLMs remains challenging; approaches include instruction fine-tuning with RLHF, prompt engineering, and security-focused refinements, among others.
• The survey highlights the unreliable correlation between intrinsic and extrinsic bias and calls for diverse benchmarks and cross-metric evaluation.