[论文解读] Towards Understanding and Mitigating Social Biases in Language Models

Paul Pu Liang, Chiyu Wu+2 more|arXiv (Cornell University)|Jun 24, 2021

Topic Modeling被引用 127

一句话总结

本论文形式化了语言模型中表征偏见的来源，提出基准测试和 Autoregressive INLP (A-INLP) 去偏方法，并证明在保持生成质量的同时实现偏见缓解。

ABSTRACT

As machine learning methods are deployed in real-world settings such as healthcare, legal systems, and social science, it is crucial to recognize how they shape social biases and stereotypes in these sensitive decision-making processes. Among such real-world deployments are large-scale pretrained language models (LMs) that can be potentially dangerous in manifesting undesirable representational biases - harmful biases resulting from stereotyping that propagate negative generalizations involving gender, race, religion, and other social constructs. As a step towards improving the fairness of LMs, we carefully define several sources of representational biases before proposing new benchmarks and metrics to measure them. With these tools, we propose steps towards mitigating social biases during text generation. Our empirical results and human evaluation demonstrate effectiveness in mitigating bias while retaining crucial contextual information for high-fidelity text generation, thereby pushing forward the performance-fairness Pareto frontier.

研究动机与目标

• Define fine-grained local and high-level global biases in language model text generation.
• Develop benchmarks and metrics that measure both local and global biases under diverse contexts.
• Propose and evaluate an autoregressive debiasing method (A-INLP) that post-processes pretrained LMs without retraining.
• Automatically identify bias-sensitive tokens to enable scalable, context-aware debiasing.
• Demonstrate bias mitigation while maintaining high-fidelity text generation on GPT-2/GPT-2-like models.

提出的方法

• Disentangle local (token-level at a time-step) versus global (entire sentence) biases in LM outputs.
• Use f-divergences (KL divergence, Hellinger distance) to quantify local biases across next-token distributions.
• Measure global biases with a pretrained sentiment/regard classifier on full generated sentences.
• Identify bias-sensitive tokens by projecting token embeddings onto a learned bias subspace derived from bias-defining word pairs.
• Apply Autoregressive INLP to remove bias information from context embeddings via a nullspace projection.
• Compute an adaptive debiasing weight alpha_t that blends debiased and original LM outputs to balance fairness and performance.

实验结果

研究问题

• RQ1 How do local and global biases manifest in language model generation?
• RQ2 Can we reliably benchmark biases using diverse real-world contexts beyond simple templates?
• RQ3 Can post-hoc, autoregressive debiasing (A-INLP) mitigate biases without retraining and with acceptable impact on language quality?
• RQ4 How can bias-sensitive tokens be automatically identified and used to guide debiasing in contextually rich generation?
• RQ5 What is the trade-off between fairness (bias mitigation) and language modeling performance when applying A-INLP?

主要发现

• Bias exists in pretrained LMs and can be characterized as local and global biases.
• A diverse-context bias classifier generalizes better to real-world contexts than classifiers trained on simple templates.
• A-INLP reduces both local and global bias metrics and often improves fairness with limited loss in language modeling performance.
• Adaptive alpha_t learning (A-INLP tune/learn) outperforms static debiasing in balancing performance and fairness on global regard tasks.
• Token-level subspace debiasing (A-subspace) can achieve fairness gains with little or no additional performance cost.
• Empirical results indicate initial debiasing can improve fairness with modest impact on generation quality, pushing the fairness-performance Pareto frontier.