[Paper Review] The Authors Matter: Understanding and Mitigating Implicit Bias in Deep Text Classification

Haochen Liu, Wei Jin+3 more|arXiv (Cornell University)|May 6, 2021

Topic Modeling30 references4 citations

TL;DR

This paper introduces Debiased-TC, a model-agnostic framework that mitigates implicit bias in deep text classifiers by training them to focus on content-based features rather than author-related linguistic patterns. By incorporating a saliency selection layer and an optimization strategy, the method reduces unfair predictions tied to author demographics while maintaining or slightly improving classification accuracy across three real-world datasets.

ABSTRACT

It is evident that deep text classification models trained on human data could be biased. In particular, they produce biased outcomes for texts that explicitly include identity terms of certain demographic groups. We refer to this type of bias as explicit bias, which has been extensively studied. However, deep text classification models can also produce biased outcomes for texts written by authors of certain demographic groups. We refer to such bias as implicit bias of which we still have a rather limited understanding. In this paper, we first demonstrate that implicit bias exists in different text classification tasks for different demographic groups. Then, we build a learning-based interpretation method to deepen our knowledge of implicit bias. Specifically, we verify that classifiers learn to make predictions based on language features that are related to the demographic attributes of the authors. Next, we propose a framework Debiased-TC to train deep text classifiers to make predictions on the right features and consequently mitigate implicit bias. We conduct extensive experiments on three real-world datasets. The results show that the text classification models trained under our proposed framework outperform traditional models significantly in terms of fairness, and also slightly in terms of classification performance.

Motivation & Objective

• To investigate the existence and mechanisms of implicit bias in deep text classification models, where predictions are unfairly influenced by author demographic attributes even when those identities are not mentioned in the text.
• To understand how deep models learn and exploit language features correlated with author demographics, leading to biased predictions.
• To develop a model-agnostic framework that corrects biased feature selection during training, thereby reducing implicit bias.
• To evaluate the effectiveness of the proposed method in improving fairness while preserving or enhancing classification performance.

Proposed method

• Propose a learning-based interpretation method to identify which language features (e.g., dialectal terms like 'goin' or 'yup') are used by models to make biased predictions based on author demographics.
• Design a saliency selection layer that explicitly guides the model to attend only to content-relevant features, filtering out demographic-correlated linguistic cues.
• Introduce a differentiable optimization method that trains the classifier to prioritize correct, content-based features during backpropagation, reducing reliance on biased signals.
• Ensure the framework is model-agnostic, allowing integration with any deep text classifier such as CNNs or transformers.
• Use data augmentation and instance weighting as baselines for comparison, but focus on learning-based correction rather than data rebalancing.

Experimental results

Research questions

• RQ1 Does implicit bias exist in deep text classifiers, where model predictions are influenced by author demographic attributes even when no identity terms appear in the text?
• RQ2 What specific linguistic features do models learn to associate with author demographics, and how do these features lead to unfair predictions?
• RQ3 Can a learning-based framework effectively identify and correct the model’s reliance on biased linguistic cues during training?
• RQ4 How does the proposed framework Debiased-TC compare to existing debiasing methods in terms of fairness and classification performance?

Key findings

• Implicit bias exists in deep text classifiers: models assign lower sentiment scores to texts written by African American authors using dialectal terms like 'yup' and 'goin', even when the sentiment is positive.
• The saliency interpretation method confirms that models learn to associate demographic-correlated language features (e.g., 'goin' for African American writers) with incorrect labels, leading to unfair predictions.
• Debiased-TC significantly reduces implicit bias across three real-world datasets, with fairness improvements observed in both accuracy and F1 scores across demographic groups.
• The framework maintains or slightly improves classification performance, with Debiased-TC achieving 63.68% accuracy and 74.05% F1 on the IMDB dataset—slightly better than the base model and competitive with data-augmented baselines.
• The method outperforms traditional debiasing techniques like data augmentation and instance weighting in fairness metrics, especially where data imbalance or identity-term swapping is not feasible.