[Paper Review] Cyberbullying Detection in Social Networks Using Deep Learning Based Models; A Reproducibility Study

Maral Dadvar, Kai Eckert|arXiv (Cornell University)|Dec 19, 2018

Hate Speech and Cyberbullying Detection17 references70 citations

TL;DR

This paper reproduces deep learning-based cyberbullying detection results on Wikipedia, Twitter, and Formspring, extends evaluation to a new YouTube dataset (~54k posts, ~4k users), and examines cross-platform transfer of models.

ABSTRACT

Cyberbullying is a disturbing online misbehaviour with troubling consequences. It appears in different forms, and in most of the social networks, it is in textual format. Automatic detection of such incidents requires intelligent systems. Most of the existing studies have approached this problem with conventional machine learning models and the majority of the developed models in these studies are adaptable to a single social network at a time. In recent studies, deep learning based models have found their way in the detection of cyberbullying incidents, claiming that they can overcome the limitations of the conventional models, and improve the detection performance. In this paper, we investigate the findings of a recent literature in this regard. We successfully reproduced the findings of this literature and validated their findings using the same datasets, namely Wikipedia, Twitter, and Formspring, used by the authors. Then we expanded our work by applying the developed methods on a new YouTube dataset (~54k posts by ~4k users) and investigated the performance of the models in new social media platforms. We also transferred and evaluated the performance of the models trained on one platform to another platform. Our findings show that the deep learning based models outperform the machine learning models previously applied to the same YouTube dataset. We believe that the deep learning based models can also benefit from integrating other sources of information and looking into the impact of profile information of the users in social networks.

Motivation & Objective

• Reproduce findings from recent literature on deep learning models for cyberbullying detection using the same datasets (Wikipedia, Twitter, Formspring).
• Extend evaluation to a new social platform (YouTube) and assess performance relative to existing approaches.
• Explore cross-platform transfer: how models trained on one platform perform on others.
• Suggest the potential benefits of incorporating additional information such as user profile data to improve detection.

Proposed method

• Replicate deep learning-based cyberbullying detection methods reported in prior literature using the same datasets (Wikipedia, Twitter, Formspring).
• Apply the reproduced methods to a new YouTube dataset (~54,000 posts from ~4,000 users) and evaluate performance.
• Compare deep learning models against traditional machine learning baselines on the YouTube dataset.
• Investigate cross-platform transfer by evaluating models trained on one platform on other platforms.
• Discuss the impact of integrating additional information, such as user profile data, on model performance.

Experimental results

Research questions

• RQ1 Do deep learning-based models outperform traditional machine learning models on the YouTube dataset as they did on earlier datasets?
• RQ2 Can the findings from Wikipedia, Twitter, and Formspring be reproduced with the same datasets?
• RQ3 How well do models trained on one social platform transfer to another platform?
• RQ4 What is the potential impact of including user profile information on cyberbullying detection performance?

Key findings

• DL-based models outperform the machine learning models previously applied to the YouTube dataset.
• The authors successfully reproduced findings from the literature on Wikipedia, Twitter, and Formspring using the same datasets.
• When applied to the new YouTube dataset, DL models showed superior performance relative to traditional ML baselines.
• Models trained on one platform can be transferred to another platform for evaluation.
• Integrating additional information, such as user profile information, may benefit model performance.