[Paper Review] Perturbation Augmentation for Fairer NLP

Rebecca Qian, Candace Ross+4 more|arXiv (Cornell University)|May 25, 2022

Topic Modeling4 citations

TL;DR

This paper introduces a neural perturbation model trained on a large human-annotated dataset (PANDA) of 98K demographic text perturbations to improve fairness in NLP. By augmenting training data with demographic attribute swaps (e.g., 'women' to 'men'), the method enhances model fairness during pretraining (FairBERTa) and fine-tuning (fairtuning), achieving improved fairness without performance degradation, validated via a new metric, fairscore, which measures robustness to demographic perturbations.

ABSTRACT

Unwanted and often harmful social biases are becoming ever more salient in NLP research, affecting both models and datasets. In this work, we ask whether training on demographically perturbed data leads to fairer language models. We collect a large dataset of human annotated text perturbations and train a neural perturbation model, which we show outperforms heuristic alternatives. We find that (i) language models (LMs) pre-trained on demographically perturbed corpora are typically more fair, and (ii) LMs finetuned on perturbed GLUE datasets exhibit less demographic bias on downstream tasks, and (iii) fairness improvements do not come at the expense of performance on downstream tasks. Lastly, we discuss outstanding questions about how best to evaluate the (un)fairness of large language models. We hope that this exploration of neural demographic perturbation will help drive more improvement towards fairer NLP.

Motivation & Objective

• To address growing concerns about social bias in NLP models that reflect and amplify harmful associations in training data.
• To overcome limitations of rule-based demographic perturbation methods, which are rigid and produce unnatural text.
• To develop a neural, controllable perturbation model trained on a large-scale human-annotated dataset of demographic text perturbations.
• To evaluate the impact of demographic data augmentation on fairness in language model pretraining and fine-tuning.
• To propose fairscore, a new extrinsic metric for measuring fairness as robustness to demographic perturbations across NLU tasks.

Proposed method

• Annotate 98,000 human-generated demographic perturbations across diverse demographic axes (e.g., gender, race) to create the PANDA dataset.
• Train a sequence-to-sequence neural model (the perturber) to generate high-quality, semantically preserved perturbations by replacing demographic terms with target attributes.
• Pretrain a large language model, FairBERTa, on a corpus augmented with perturbed examples to reduce demographic bias during pretraining.
• Fine-tune existing models on perturbation-augmented GLUE datasets using a method called 'fairtuning' to improve fairness on downstream tasks.
• Propose fairscore, an extrinsic fairness metric that computes the change in model predictions between original and perturbed inputs to quantify robustness to demographic shifts.

Experimental results

Research questions

• RQ1 Can neural demographic perturbation improve fairness in pre-trained language models without sacrificing performance?
• RQ2 Does fine-tuning on perturbation-augmented datasets (fairtuning) lead to reduced demographic bias in downstream NLU tasks?
• RQ3 How effective is the proposed fairscore metric in measuring fairness as robustness to demographic perturbations across diverse NLP tasks?
• RQ4 To what extent do demographic perturbations alter the original gold labels in standard NLU datasets?
• RQ5 Can a large-scale, human-annotated perturbation dataset (PANDA) enable training of higher-quality neural perturbers than heuristic rule-based systems?

Key findings

• Language models pre-trained on demographically perturbed corpora (e.g., FairBERTa) exhibit significantly improved fairness without performance degradation on downstream tasks.
• Fine-tuning models on perturbation-augmented GLUE datasets (fairtuning) results in models that are fairer on average than those fine-tuned on original, unperturbed data.
• The neural perturber outperforms heuristic alternatives in generating natural, semantically preserved perturbations, as validated by human evaluation.
• The fairscore metric successfully quantifies fairness as robustness to demographic perturbations and shows significant improvements after fairtuning across multiple GLUE tasks.
• Pilot validation found that only 4–5 out of 25 examples per task (on average) had label disagreements after perturbation, indicating that the core assumption of label invariance holds in most cases.
• The study demonstrates that data-level augmentation via neural demographic perturbation is a viable and effective strategy for reducing bias in NLP models.