[Paper Review] Hate Speech Detection from Code-mixed Hindi-English Tweets Using Deep Learning Models

Kamble Satyajit, Aditya Joshi|arXiv (Cornell University)|Nov 13, 2018

Hate Speech and Cyberbullying Detection13 references74 citations

TL;DR

The paper develops three deep learning models (CNN-1D, LSTM, BiLSTM) with domain-specific embeddings to detect hate speech in Hindi-English code-mixed tweets and shows a ~12% F-score improvement over a statistical baseline.

ABSTRACT

This paper reports an increment to the state-of-the-art in hate speech detection for English-Hindi code-mixed tweets. We compare three typical deep learning models using domain-specific embeddings. On experimenting with a benchmark dataset of English-Hindi code-mixed tweets, we observe that using domain-specific embeddings results in an improved representation of target groups, and an improved F-score.

Motivation & Objective

• Motivate hate speech detection in Hindi-English code-mixed tweets in a social media context.
• Evaluate deep learning architectures against a statistical baseline on a common dataset.
• Show that domain-specific word embeddings improve representation of target groups in hate speech.
• Compare CNN-1D, LSTM, and BiLSTM to identify the best-performing model.

Proposed method

• Train domain-specific word embeddings from a large code-mixed tweet corpus using gensim Word2Vec.
• Implement three deep learning models (CNN-1D, LSTM, BiLSTM) with domain-specific embeddings for binary hate speech classification.
• Use global max pooling and dropout in CNN-1D, and sequence processing with LSTM/BiLSTM followed by global max pooling.
• Train and evaluate models with 10-fold cross-validation on the Bohra et al. (2018) dataset.
• Re-implement the baseline statistical approaches (SVM, Random Forest) for fair comparison.

Experimental results

Research questions

• RQ1 Can deep learning models with domain-specific embeddings outperform traditional statistical methods on Hindi-English code-mixed hate speech datasets?
• RQ2 Which neural architecture (CNN-1D, LSTM, BiLSTM) best captures semantics in code-mixed hate speech?
• RQ3 Do domain-specific embeddings better represent minority groups and hate expressions than general embeddings?
• RQ4 What are the precision, recall, F-score, and accuracy gains of deep learning models over the statistical baseline on the target dataset.

Key findings

• CNN-1D achieved the highest performance with a F-score of 80.85% and an accuracy of 82.62%.
• The deep learning models collectively outperformed the statistical baseline in F-score, precision, and recall.
• Domain-specific embeddings improve similarity representations of minority groups and hate expressions compared to general embeddings.
• BiLSTM offers slightly better recall than CNN-1D, while CNN-1D provides higher precision.
• Domain-specific embeddings contain swear words not present in general embeddings, improving semantic representations for hate speech.
• The improvement over the statistical baseline is about 12% in F-score.