[Paper Review] All You Need is "Love": Evading Hate-speech Detection

Tommi Gröndahl, Luca Pajola+3 more|arXiv (Cornell University)|Aug 28, 2018

Hate Speech and Cyberbullying Detection73 citations

TL;DR

The paper reproduces seven state-of-the-art hate speech detectors, shows performance is dataset-dependent, and demonstrates robust adversarial attacks that break word-based models more than character-based ones, with partial mitigation via adversarial training and preprocessing tweaks.

ABSTRACT

With the spread of social networks and their unfortunate use for hate speech, automatic detection of the latter has become a pressing problem. In this paper, we reproduce seven state-of-the-art hate speech detection models from prior work, and show that they perform well only when tested on the same type of data they were trained on. Based on these results, we argue that for successful hate speech detection, model architecture is less important than the type of data and labeling criteria. We further show that all proposed detection techniques are brittle against adversaries who can (automatically) insert typos, change word boundaries or add innocuous words to the original hate speech. A combination of these methods is also effective against Google Perspective -- a cutting-edge solution from industry. Our experiments demonstrate that adversarial training does not completely mitigate the attacks, and using character-level features makes the models systematically more attack-resistant than using word-level features.

Motivation & Objective

• Assess transferability of hate speech detection models across multiple datasets.
• Evaluate whether model architecture or data labeling drives detection performance.
• Identify adversarial text modifications that can evade hate speech detectors.
• Explore simple defenses against adversarial text attacks and their limitations.

Proposed method

• Replicate seven hate speech detection models from four prior works across four datasets.
• Evaluate cross-dataset generalization by training on one dataset and testing on others.
• Systematically implement six adversarial text attacks (three categories with two variants each).
• Test adversarial training and preprocessing-based mitigations (spell checking, space handling).
• Compare word-based versus character-based models in vulnerability to attacks.

Experimental results

Research questions

• RQ1 Do hate speech detectors generalize well across different datasets and domains?
• RQ2 Is performance dominated by model architecture or the data/labeling criteria?
• RQ3 Can simple automatic text modifications evade detectors, and are some model types more robust than others?
• RQ4 Do adversarial training and preprocessing mitigations reduce attack effectiveness?
• RQ5 How do word-based and character-based models differ in susceptibility to specific attack types?

Key findings

• All models perform similarly when trained and tested on the same dataset, but transfer across datasets fails notably, showing strong dataset dependence.
• Word-based models are more vulnerable to tokenization/word-boundary attacks than character-based models.
• Character-based models are not completely breakable by attacks, and adversarial training substantially improves robustness, especially against leetspeak.
• Word-appending attacks can degrade both word- and character-based models, though mitigation effectiveness varies by method.
• Pre-trained language transfer (ULMFiT) did not outperform baselines in this replication.
• Google Perspective scores (toxicity) are easily inflated by simple non-hateful additions, highlighting mismatch with hate speech.