[Paper Review] Demoting Racial Bias in Hate Speech Detection

Mengzhou Xia, Anjalie Field+1 more|arXiv (Cornell University)|May 25, 2020

Hate Speech and Cyberbullying Detection4 citations

TL;DR

This paper proposes an adversarial training method to reduce racial bias in hate speech detection by demoting reliance on African American English (AAE) features in model predictions. By training a classifier to predict toxicity while minimizing information about AAE through a dedicated adversary, the approach reduces false positive rates for AAE text by 2.2–3.2% without sacrificing overall classification performance.

ABSTRACT

In current hate speech datasets, there exists a high correlation between annotators' perceptions of toxicity and signals of African American English (AAE). This bias in annotated training data and the tendency of machine learning models to amplify it cause AAE text to often be mislabeled as abusive/offensive/hate speech with a high false positive rate by current hate speech classifiers. In this paper, we use adversarial training to mitigate this bias, introducing a hate speech classifier that learns to detect toxic sentences while demoting confounds corresponding to AAE texts. Experimental results on a hate speech dataset and an AAE dataset suggest that our method is able to substantially reduce the false positive rate for AAE text while only minimally affecting the performance of hate speech classification.

Motivation & Objective

• To address the high false positive rate in hate speech detection for African American English (AAE) text, which stems from biased crowd annotations.
• To mitigate the amplification of AAE-toxicity spurious correlations by machine learning models trained on biased datasets.
• To develop a method that preserves hate speech detection performance while reducing model dependence on dialect-related features.
• To achieve fairness in NLP by demoting protected attributes (like AAE) during model training, aligning with the Equality of Opportunity criterion.
• To provide a generalizable framework for debiasing text classifiers in any task where protected attributes may introduce unfairness.

Proposed method

• The model uses a three-part architecture: a bidirectional LSTM encoder with attention, a binary toxicity classifier, and a dedicated adversary network for AAE detection.
• Adversarial training is employed to minimize the information about AAE in the encoded text representation, encouraging the classifier to focus on toxicity cues instead.
• The training objective combines standard cross-entropy loss for the toxicity classifier and adversarial loss to discourage the adversary from accurately predicting AAE from the encoded representation.
• The model is trained on datasets where AAE is inferred via demographic information (Blodgett et al., 2016) or automatically detected in hate speech datasets (Founta et al., 2018).
• Validation accuracy of the adversary is monitored to ensure dialectal information is progressively removed from the representation.
• Multiple adversaries are evaluated but found not to improve performance over a single adversary in this single-attribute demotion setting.

Experimental results

Research questions

• RQ1 Can adversarial training effectively reduce the model’s reliance on African American English (AAE) features in hate speech detection?
• RQ2 To what extent can the false positive rate (FPR) for AAE text be reduced without degrading overall hate speech classification performance?
• RQ3 Does the proposed method achieve a measurable reduction in bias while preserving the utility of the classifier?
• RQ4 How does the use of a single adversary compare to multiple adversaries in demoting dialectal information in this context?
• RQ5 Can this approach generalize to other text classification tasks where protected attributes introduce unfairness?

Key findings

• The proposed method reduces the false positive rate (FPR) for AAE text by 2.2–3.2 percentage points on evaluated datasets, significantly lowering misclassification of non-abusive AAE content.
• The validation accuracy of the AAE adversary decreases by 6–10 points (DWMW17) and 2–5 points (FDCL18) over training, confirming progressive demotion of dialectal information.
• Despite reduced AAE prediction accuracy, the main toxicity classifier maintains strong performance, indicating minimal degradation in core task utility.
• Using multiple adversaries did not improve results over a single adversary, suggesting that multiple adversaries are unnecessary in this single-attribute demotion setting.
• The model successfully learns a representation that is partially invariant to AAE, reducing bias while preserving predictive power for toxicity.
• The method demonstrates potential for broader application in debiasing NLP models across various protected attributes and classification tasks.