[Paper Review] Prompting Fairness: Integrating Causality to Debias Large Language Models

Jingling Li, Zeyu Tang+5 more|arXiv (Cornell University)|Mar 13, 2024

Legal Education and Practice InnovationsSocial Sciences6 citations

TL;DR

The paper proposes a causality-guided debiasing framework for LLMs that leverages data-generation and reasoning-process causality to design prompts that discourage biased reasoning and encourage bias-free reasoning, achieving strong empirical debiasing on WinoBias and Discrim-Eval with black-box access.

ABSTRACT

Large language models (LLMs), despite their remarkable capabilities, are susceptible to generating biased and discriminatory responses. As LLMs increasingly influence high-stakes decision-making (e.g., hiring and healthcare), mitigating these biases becomes critical. In this work, we propose a causality-guided debiasing framework to tackle social biases, aiming to reduce the objectionable dependence between LLMs' decisions and the social information in the input. Our framework introduces a novel perspective to identify how social information can affect an LLM's decision through different causal pathways. Leveraging these causal insights, we outline principled prompting strategies that regulate these pathways through selection mechanisms. This framework not only unifies existing prompting-based debiasing techniques, but also opens up new directions for reducing bias by encouraging the model to prioritize fact-based reasoning over reliance on biased social cues. We validate our framework through extensive experiments on real-world datasets across multiple domains, demonstrating its effectiveness in debiasing LLM decisions, even with only black-box access to the model.

Motivation & Objective

• Model and mitigate social biases in LLM outputs by analyzing how demographic information can spur biased reasoning via selection mechanisms.
• Develop a principled prompting framework grounded in causal models of data generation and model reasoning.
• Unify existing debiasing prompts (inhibitive and contrastive) under a causal debiasing strategy and evaluate across closed- and open-access models.
• Provide empirically robust guidance for debiasing LLMs with only black-box access.

Proposed method

• Construct causal models of the training data generation process to identify how demographic information can be associated with biased outputs.
• Construct a causal model of LLM reasoning and connect it to the data-gen model through selection mechanisms modulated by prompts.
• Propose three prompting strategies (Strategy I–III) that impose conditions on internal representations and selection paths to debias outputs.
• Formalize conditions that prompt designs should satisfy to nudge towards demographic-agnostic facts and counteract existing bias.
• Empirically evaluate debiasing strategies on WinoBias and Discrim-Eval, comparing against baselines (Default, ICL with contrastive examples, Zero-shot COT).
• Show that combining encouraging bias-free reasoning with discouraging biased reasoning yields stronger debiasing than baselines.

Experimental results

Research questions

• RQ1 How can causal models of data generation and LLM reasoning explain the emergence of demographic biases in outputs?
• RQ2 Can prompt designs be crafted to control selection mechanisms in LLMs to reduce bias under black-box access?
• RQ3 Do strategies that encourage bias-free reasoning and/or discourage biased reasoning outperform traditional prompting baselines in debiasing LLMs?
• RQ4 What is the empirical impact of causality-guided debiasing on gender bias across coreference tasks and real-world datasets?

Key findings

• Prompts that encourage bias-free reasoning and discourage biased reasoning substantially reduce gender bias across LLMs on WinoBias, including large gaps between pro- and anti-sentences.
• Combined Reduce + Fact approach achieves the smallest bias gaps, with GPT-4 showing a 2.17% gap for Type I and 0.13% gap for Type II coreference tasks in certain settings.
• On Discrim-Eval, prompting strategies universal reduce discrimination across demographics, with higher-capability models showing reduced bias gaps.
• The framework unifies existing prompting-based debiasing methods by interpreting them as instances of the proposed causal prompt-design strategies.
• The results hold with black-box access, demonstrating practical applicability for closed-source LLMs.