[Paper Review] Trawling for Trolling: A Dataset

Hitkul, Karmanya Aggarwal+4 more|arXiv (Cornell University)|Aug 2, 2020

Marine and fisheries research4 citations

TL;DR

This paper introduces a publicly available, manually curated dataset of 12,490 social media samples labeled across five classes: Normal, Profanity, Trolling, Derogatory, and Hate Speech. By re-annotating existing datasets with precise definitions, the authors create a resource that distinguishes trolling from hate speech, enabling models to learn meaningful linguistic patterns rather than statistical artifacts, as shown by strong performance and sensitivity to data ablation.

ABSTRACT

The ability to accurately detect and filter offensive content automatically is important to ensure a rich and diverse digital discourse. Trolling is a type of hurtful or offensive content that is prevalent in social media, but is underrepresented in datasets for offensive content detection. In this work, we present a dataset that models trolling as a subcategory of offensive content. The dataset was created by collecting samples from well-known datasets and reannotating them along precise definitions of different categories of offensive content. The dataset has 12,490 samples, split across 5 classes; Normal, Profanity, Trolling, Derogatory and Hate Speech. It encompasses content from Twitter, Reddit and Wikipedia Talk Pages. Models trained on our dataset show appreciable performance without any significant hyperparameter tuning and can potentially learn meaningful linguistic information effectively. We find that these models are sensitive to data ablation which suggests that the dataset is largely devoid of spurious statistical artefacts that could otherwise distract and confuse classification models.

Motivation & Objective

• To address the lack of datasets that distinguish trolling from other forms of offensive content, particularly hate speech.
• To create a publicly available, manually annotated dataset that captures the nuances of trolling as a distinct category of offensive content.
• To reduce spurious statistical artifacts in offensive content detection datasets by rigorous data validation and ablation testing.
• To improve the fairness and accuracy of automated content moderation by enabling models to differentiate between mild, disruptive trolling and severe hate speech.

Proposed method

• The dataset was constructed by re-annotating samples from previously released public offensive content datasets using clearly defined, empirically grounded categories.
• Each sample was labeled by human annotators using precise definitions for five classes: Normal, Profanity, Trolling, Derogatory, and Hate Speech.
• The dataset spans three platforms: Twitter, Reddit, and Wikipedia Talk Pages, ensuring cross-platform linguistic diversity.
• Model evaluation used BERT-based classifiers trained on the full dataset with no significant hyperparameter tuning to assess generalization and linguistic learning.
• Data ablation tests—shuffling labels, scrambling word order, and removing tokens—were applied to evaluate whether models rely on linguistic structure or spurious cues.
• Statistical analysis of top n-gram cues per class was performed to verify the absence of strong unigram or short-sequence biases that could mislead models.

Experimental results

Research questions

• RQ1 Can a dataset that explicitly distinguishes trolling from hate speech improve the performance and fairness of offensive content detection models?
• RQ2 To what extent do state-of-the-art models learn meaningful linguistic patterns rather than relying on superficial statistical cues in offensive content detection?
• RQ3 Does the absence of strong unigram or n-gram cues in the dataset reduce model dependence on spurious statistical artifacts?
• RQ4 How sensitive are models to data ablation when trained on this dataset, indicating their reliance on contextual and sequential information?
• RQ5 Can this dataset serve as a benchmark for evaluating the robustness and linguistic generalization of NLP models in offensive language detection?

Key findings

• Models trained on the proposed dataset achieved 75.3% accuracy and 0.73 weighted F1 score without hyperparameter tuning, indicating strong baseline performance.
• The model's performance dropped by 35.1 percentage points when labels were shuffled, indicating it learned to associate cues with correct labels rather than memorizing data.
• Scrambling word order reduced accuracy to 62.4%, showing the model relies on sequential structure rather than bag-of-words representations.
• Removing 50% of tokens reduced accuracy to 60.8%, confirming that models use information from the full sequence, not just isolated keywords.
• Top n-gram cues for each class showed no dominant unigram patterns, suggesting the dataset avoids strong statistical biases that could mislead models.
• The dataset's design and validation process result in models that learn contextual and linguistic features, making it suitable for training robust, fair, and interpretable offensive content detection systems.