[Paper Review] Emotion Detection From Social Media Posts

Md Mahbubur Rahman, Shaila Shova|arXiv (Cornell University)|Feb 11, 2023

Sentiment Analysis and Opinion Mining4 citations

TL;DR

This study proposes a hybrid deep learning model combining BiGRU and BiLSTM for classifying emotions in social media texts into four categories: fear, anger, joy, and sadness. Using a preprocessed Twitter dataset of 7,102 samples, the ensemble model achieved 87.66% accuracy, outperforming traditional machine learning models and demonstrating strong generalization with minimal overfitting.

ABSTRACT

Over the last few years, social media has evolved into a medium for expressing personal views, emotions, and even business and political proposals, recommendations, and advertisements. We address the topic of identifying emotions from text data obtained from social media posts like Twitter in this research. We have deployed different traditional machine learning techniques such as Support Vector Machines (SVM), Naive Bayes, Decision Trees, and Random Forest, as well as deep neural network models such as LSTM, CNN, GRU, BiLSTM, BiGRU to classify these tweets into four emotion categories (Fear, Anger, Joy, and Sadness). Furthermore, we have constructed a BiLSTM and BiGRU ensemble model. The evaluation result shows that the deep neural network models(BiGRU, to be specific) produce the most promising results compared to traditional machine learning models, with an 87.53 % accuracy rate. The ensemble model performs even better (87.66 %), albeit the difference is not significant. This result will aid in the development of a decision-making tool that visualizes emotional fluctuations.

Motivation & Objective

• To develop an accurate machine learning model for classifying four primary emotions—fear, anger, joy, and sadness—in social media text.
• To compare the performance of traditional machine learning models (SVM, Naive Bayes, Decision Tree, Random Forest) with deep neural networks (LSTM, CNN, GRU, BiLSTM, BiGRU) on emotion classification.
• To design and evaluate an ensemble model combining BiGRU and BiLSTM for improved emotion detection accuracy.
• To support mental health monitoring and personalized content recommendation by visualizing emotional trends over time.
• To identify and mitigate class imbalance effects through model evaluation and hyperparameter tuning.

Proposed method

• Collected a labeled Twitter dataset of 7,102 English-language posts from Kaggle, categorized into four emotion classes: anger, fear, joy, and sadness.
• Performed extensive text pre-processing including lowercasing, removal of URLs, HTML tags, punctuation, stop words, and expansion of contractions and abbreviations.
• Applied lemmatization and stemming to reduce lexical variation and normalize word forms while preserving emoticons for emotional context.
• Trained and evaluated multiple models: traditional ML (SVM, Naive Bayes, Decision Tree, Random Forest, Logistic Regression) and deep learning models (LSTM, CNN, GRU, BiLSTM, BiGRU).
• Constructed an ensemble model by combining predictions from BiGRU and BiLSTM, using weighted averaging or voting to improve generalization.
• Optimized hyperparameters for each model using cross-validation and selected the best-performing configuration based on validation accuracy.

Experimental results

Research questions

• RQ1 How do traditional machine learning models compare to deep neural networks in classifying emotions from short social media texts?
• RQ2 What is the impact of model architecture (e.g., BiGRU vs. BiLSTM) on emotion classification performance on a small, imbalanced dataset?
• RQ3 Can an ensemble of deep learning models improve generalization and reduce overfitting compared to individual models?
• RQ4 How do class distribution imbalances affect precision, recall, and F1-score across emotion categories?
• RQ5 To what extent does the inclusion of emoticons and pre-processing techniques influence model performance in emotion detection?

Key findings

• The BiGRU model achieved the highest individual accuracy at 87.53%, outperforming all traditional machine learning models.
• The ensemble model of BiGRU and BiLSTM achieved the best overall accuracy of 87.66%, demonstrating a marginal but consistent improvement over individual deep learning models.
• Naive Bayes performed poorly (accuracy ~42%) due to its assumption of feature independence, which does not hold in sequential text data.
• The nearest neighbor classifier underperformed due to the curse of dimensionality, as high-dimensional feature vectors reduced distance-based discrimination.
• Class sadness had the lowest F1-score (0.82) due to high false positives—51 fear and 37 anger samples were misclassified as sadness—indicating class confusion.
• The micro- and macro-averaged AUC values were approximately 0.97, indicating strong overall model performance despite class-specific limitations.