[Paper Review] Towards non-toxic landscapes: Automatic toxic comment detection using DNN

Ashwin Geet d'Sa, Irina Illina+1 more|arXiv (Cornell University)|Nov 19, 2019

Hate Speech and Cyberbullying DetectionComputer Science24 references19 citations

TL;DR

This paper proposes a deep neural network (DNN)-based approach for automatic toxic comment detection using BERT fine-tuning, outperforming feature-based BERT, fastText, and Mikolov embeddings. It evaluates robustness against adversarial perturbations and demonstrates state-of-the-art performance on the English Wikipedia Detox dataset, establishing BERT fine-tuning as the most effective method for classifying toxic content.

ABSTRACT

The spectacular expansion of the Internet has led to the development of a new research problem in the field of natural language processing: automatic toxic comment detection, since many countries prohibit hate speech in public media. There is no clear and formal definition of hate, offensive, toxic and abusive speeches. In this article, we put all these terms under the umbrella of "toxic" speech. The contribution of this paper is the design of binary classification and regression-based approaches aiming to predict whether a comment is toxic or not. We compare different unsupervised word representations and different DNN based classifiers. Moreover, we study the robustness of the proposed approaches to adversarial attacks by adding one (healthy or toxic) word. We evaluate the proposed methodology on the English Wikipedia Detox corpus. Our experiments show that using BERT fine-tuning outperforms feature-based BERT, Mikolov's and fastText representations with different DNN classifiers.

Motivation & Objective

• To develop a robust and accurate method for classifying toxic comments in online discourse.
• To compare the effectiveness of various word representations (e.g., BERT, fastText, Mikolov) in toxic comment detection.
• To evaluate the robustness of DNN models against adversarial word perturbations (toxic or benign).
• To establish a strong baseline for toxic language detection using deep learning on real-world web text.

Proposed method

• The authors employ binary classification and regression-based DNN models to predict toxicity in comments.
• They compare multiple unsupervised word representations: BERT (fine-tuned and feature-based), fastText, and Mikolov’s skip-gram.
• Models are trained and evaluated on the English Wikipedia Detox dataset, a standard benchmark for toxic comment detection.
• Adversarial robustness is tested by inserting one word (either toxic or non-toxic) into comments and measuring model performance shifts.
• Fine-tuned BERT is used as the primary model, leveraging contextual embeddings for improved semantic understanding.
• The study uses standard NLP evaluation metrics such as AUC-ROC and F1-score to compare model performance.

Experimental results

Research questions

• RQ1 Does fine-tuned BERT outperform other word representation methods in toxic comment detection?
• RQ2 How do different DNN architectures perform when using various pre-trained embeddings?
• RQ3 To what extent are DNN models robust to adversarial word insertions in comments?
• RQ4 Can a unified framework effectively classify diverse forms of toxic language, including hate speech and offensive content?

Key findings

• Fine-tuned BERT achieves the highest performance among all models tested, significantly outperforming feature-based BERT, fastText, and Mikolov embeddings.
• The model maintains high robustness when a single word is added, indicating resilience to minor adversarial perturbations.
• Feature-based BERT shows strong performance but underperforms compared to fine-tuned BERT, highlighting the benefit of fine-tuning on downstream tasks.
• FastText and Mikolov embeddings yield lower performance, especially in capturing nuanced toxic language.
• The regression-based approach shows comparable results to binary classification, suggesting flexibility in task formulation.