[Paper Review] Bias and Fairness in Large Language Models: A Survey

Isabel O. Gallegos, Ryan A. Rossi+7 more|arXiv (Cornell University)|Sep 2, 2023

Text Readability and Simplification58 citations

TL;DR

This survey consolidates definitions of social bias and fairness in LLMs, introduces taxonomies for bias evaluation metrics and datasets, and classifies bias mitigation techniques across pre-processing, in-training, intra-processing, and post-processing.

ABSTRACT

Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touch our social sphere. Despite this success, these models can learn, perpetuate, and amplify harmful social biases. In this paper, we present a comprehensive survey of bias evaluation and mitigation techniques for LLMs. We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing, defining distinct facets of harm and introducing several desiderata to operationalize fairness for LLMs. We then unify the literature by proposing three intuitive taxonomies, two for bias evaluation, namely metrics and datasets, and one for mitigation. Our first taxonomy of metrics for bias evaluation disambiguates the relationship between metrics and evaluation datasets, and organizes metrics by the different levels at which they operate in a model: embeddings, probabilities, and generated text. Our second taxonomy of datasets for bias evaluation categorizes datasets by their structure as counterfactual inputs or prompts, and identifies the targeted harms and social groups; we also release a consolidation of publicly-available datasets for improved access. Our third taxonomy of techniques for bias mitigation classifies methods by their intervention during pre-processing, in-training, intra-processing, and post-processing, with granular subcategories that elucidate research trends. Finally, we identify open problems and challenges for future work. Synthesizing a wide range of recent research, we aim to provide a clear guide of the existing literature that empowers researchers and practitioners to better understand and prevent the propagation of bias in LLMs.

Motivation & Objective

• Consolidate and formalize social bias and fairness notions for NLP and LLMs.
• Develop taxonomies organizing bias evaluation metrics by data structure and model access.
• Compile and categorize publicly available bias evaluation datasets for LLMs.
• Classify bias mitigation techniques by intervention stage and provide unified notation for methods.
• Identify open problems and challenges to guide future research in fair LLMs.

Proposed method

• Formalize LLM concepts and fairness desiderata tailored to NLP and LLMs.
• Propose three taxonomies: (i) bias evaluation metrics (embeddings, probabilities, generated text), (ii) bias evaluation datasets (counterfactual inputs, prompts), (iii) bias mitigation techniques (pre-, in-, intra-, post-processing).
• Provide unified mathematical notation to compare metrics and formalize techniques.
• Consolidate and release publicly available datasets for bias evaluation.
• Discuss open problems and future directions for reducing bias in LLMs.

Experimental results

Research questions

• RQ1 What are the precise facets of social bias and fairness relevant to LLMs and NLP tasks?
• RQ2 How can bias evaluation metrics be organized by data structure and model access to enable consistent assessment?
• RQ3 What datasets exist for bias evaluation and how can they be standardized or consolidated?
• RQ4 What taxonomy best describes bias mitigation techniques across intervention stages?
• RQ5 What are the key open challenges and future directions for achieving fairness in LLMs?

Key findings

• The paper provides formal definitions of social bias, group and individual fairness, and a taxonomy of harms (representational and allocational) applicable to NLP and LLMs.
• It offers a unified taxonomy of bias evaluation metrics across embeddings, probabilities, and generated text, clarifying the link between metrics and evaluation datasets.
• It consolidates datasets for bias evaluation by structure (counterfactual inputs, prompts) and documents targeted harms and social groups, with a public repository for access.
• It presents a taxonomy of mitigation techniques organized by intervention stage (pre-, in-, intra-, post-processing) with granular subcategories and formalization.
• The survey highlights open problems, including robustness of fairness notions, evaluation standards, and expansion of mitigation efforts across the NLP lifecycle.