[Paper Review] Hate is not Binary: Studying Abusive Behavior of #GamerGate on Twitter

Despoina Chatzakou, Nicolas Kourtellis+4 more|arXiv (Cornell University)|May 9, 2017

Hate Speech and Cyberbullying Detection22 citations

TL;DR

This study analyzes abusive behavior in the #GamerGate Twitter controversy, revealing that Gamergaters exhibit less joy than typical users despite aggressive content. Using unsupervised clustering and supervised classification, the authors identify behavioral features that predict suspension, achieving high precision and recall, and uncover why abusive users evade detection despite harmful activity.

ABSTRACT

Over the past few years, online bullying and aggression have become increasingly prominent, and manifested in many different forms on social media. However, there is little work analyzing the characteristics of abusive users and what distinguishes them from typical social media users. In this paper, we start addressing this gap by analyzing tweets containing a great large amount of abusiveness. We focus on a Twitter dataset revolving around the Gamergate controversy, which led to many incidents of cyberbullying and cyberaggression on various gaming and social media platforms. We study the properties of the users tweeting about Gamergate, the content they post, and the differences in their behavior compared to typical Twitter users. We find that while their tweets are often seemingly about aggressive and hateful subjects, "Gamergaters" do not exhibit common expressions of online anger, and in fact primarily differ from typical users in that their tweets are less joyful. They are also more engaged than typical Twitter users, which is an indication as to how and why this controversy is still ongoing. Surprisingly, we find that Gamergaters are less likely to be suspended by Twitter, thus we analyze their properties to identify differences from typical users and what may have led to their suspension. We perform an unsupervised machine learning analysis to detect clusters of users who, though currently active, could be considered for suspension since they exhibit similar behaviors with suspended users. Finally, we confirm the usefulness of our analyzed features by emulating the Twitter suspension mechanism with a supervised learning method, achieving very good precision and recall.

Motivation & Objective

• To understand the behavioral differences between abusive Gamergaters and typical Twitter users.
• To investigate why Gamergaters are disproportionately not suspended by Twitter despite aggressive content.
• To identify behavioral clusters of active users who resemble suspended accounts and should be considered for suspension.
• To evaluate the performance of supervised models in emulating Twitter’s suspension mechanism.

Proposed method

• Collected and cleaned a large-scale Twitter dataset centered on the #GamerGate controversy.
• Performed comparative analysis of user behavior, sentiment, and network engagement between Gamergaters and random Twitter users.
• Applied unsupervised machine learning to detect clusters of active users with suspension-like behaviors.
• Trained supervised classifiers using emotional and activity-related features to predict user status (active, deleted, suspended).
• Evaluated model performance using precision, recall, and ROC-AUC metrics on both GG and baseline datasets.
• Analyzed account deletion patterns and emotional profiles of users who self-deleted their accounts.

Experimental results

Research questions

• RQ1 How do the behavioral patterns of Gamergaters differ from those of typical Twitter users?
• RQ2 Why are Gamergaters less likely to be suspended by Twitter despite posting abusive content?
• RQ3 What features distinguish suspended users from active or deleted users in the context of Gamergate?
• RQ4 Can unsupervised clustering identify active users with behaviors similar to suspended users?
• RQ5 How well can a supervised model predict user suspension status based on behavioral and emotional features?

Key findings

• Gamergaters differ from typical users primarily in expressing less joy, despite posting aggressive and hateful content.
• Gamergaters are more engaged than typical users, with higher numbers of friends and followers, contributing to the controversy's longevity.
• Gamergaters are disproportionately less likely to be suspended than random users, despite posting offensive content.
• Suspended Gamergaters exhibit more aggressive emotions, offensive language, and higher levels of joy than suspended random users.
• Users who deleted their accounts showed the highest activity levels, signs of distress, fear, and sadness, and had small, unsupportive social networks.
• Supervised models achieved high performance in predicting suspension status, with average F1-scores above 0.88 and AUC values over 0.88 on the GG dataset.