[Paper Review] Should We Attend More or Less? Modulating Attention for Fairness

Abdelrahman Zayed, Gonçalo Mordido+2 more|arXiv (Cornell University)|May 22, 2023

Topic Modeling4 citations

TL;DR

This paper proposes EAT (Entropy-based Attention Temperature Scaling), a novel intra-processing method that modulates attention distribution entropy post-training to improve fairness in NLP models with minimal performance loss. By dynamically adjusting the temperature of attention logits, EAT enhances fairness across gender, race, religion, and other social biases in both text classification and generation tasks, outperforming existing methods with under 3.5% accuracy degradation.

ABSTRACT

The advances in natural language processing (NLP) pose both opportunities and challenges. While recent progress enables the development of high-performing models for a variety of tasks, it also poses the risk of models learning harmful biases from the data, such as gender stereotypes. In this work, we investigate the role of attention, a widely-used technique in current state-of-the-art NLP models, in the propagation of social biases. Specifically, we study the relationship between the entropy of the attention distribution and the model's performance and fairness. We then propose a novel method for modulating attention weights to improve model fairness after training. Since our method is only applied post-training and pre-inference, it is an intra-processing method and is, therefore, less computationally expensive than existing in-processing and pre-processing approaches. Our results show an increase in fairness and minimal performance loss on different text classification and generation tasks using language models of varying sizes. WARNING: This work uses language that is offensive.

Motivation & Objective

• To investigate the relationship between attention entropy and model fairness in NLP models.
• To address gender and other social biases in fine-tuned BERT and RoBERTa models without retraining.
• To develop a computationally efficient intra-processing method that modifies attention distributions after training.
• To evaluate generalization of the method across diverse social biases beyond just gender.
• To extend the method to text generation tasks using GPT-Neo.

Proposed method

• EAT applies temperature scaling to the attention logits post-training to modulate the entropy of the attention distribution.
• The temperature hyperparameter is tuned exclusively on gender bias using validation data.
• The method is applied during inference, making it lightweight and compatible with any pre-trained model.
• EAT is combined with existing in-processing and pre-processing debiasing methods to assess additive fairness improvements.
• The approach is evaluated on three text classification datasets and extended to text generation using the BOLD benchmark.
• Sentiment analysis via VADER measures bias in model outputs, with perplexity used to assess generation quality.

Experimental results

Research questions

• RQ1 How does attention distribution entropy relate to fairness and performance in fine-tuned BERT and RoBERTa models?
• RQ2 Can post-training modulation of attention entropy improve fairness without significant performance degradation?
• RQ3 Does the proposed EAT method generalize to social biases beyond gender, such as race, religion, and sexual orientation?
• RQ4 How does EAT compare to other intra-processing, in-processing, and pre-processing debiasing methods in fairness and robustness?
• RQ5 Can EAT be effectively applied to text generation models like GPT-Neo to mitigate social bias?

Key findings

• EAT achieved the highest fairness ranking across all tested bias types, outperforming other intra-processing methods in 83% of scenarios.
• The method caused less than 3.5% performance degradation on all text classification tasks, with random perturbation being the only exception.
• EAT reduced negative sentiment in model generations across all demographic groups, while maintaining low perplexity.
• The hyperparameters tuned solely on gender bias generalized effectively to other social biases, including race, religion, and disability.
• Combining EAT with existing in-processing and pre-processing methods consistently improved fairness across all settings.
• EAT demonstrated robustness even when validation and test templates had different data distributions.