[Paper Review] Emotion Detection in Text: Focusing on Latent Representation

Armin Seyeditabari, Narges Tabari+2 more|arXiv (Cornell University)|Jul 22, 2019

Sentiment Analysis and Opinion Mining26 references10 citations

TL;DR

This paper proposes a bidirectional GRU-based deep learning model that improves emotion detection in text by learning more informative latent representations, capturing sequential and contextual nuances. The method achieves a 26.8-point average F1-measure improvement over baseline models on a tweet dataset and a 38.6-point increase on a new, noisier dataset, demonstrating superior generalization and robustness across emotion categories.

ABSTRACT

In recent years, emotion detection in text has become more popular due to its vast potential applications in marketing, political science, psychology, human-computer interaction, artificial intelligence, etc. In this work, we argue that current methods which are based on conventional machine learning models cannot grasp the intricacy of emotional language by ignoring the sequential nature of the text, and the context. These methods, therefore, are not sufficient to create an applicable and generalizable emotion detection methodology. Understanding these limitations, we present a new network based on a bidirectional GRU model to show that capturing more meaningful information from text can significantly improve the performance of these models. The results show significant improvement with an average of 26.8 point increase in F-measure on our test data and 38.6 increase on the totally new dataset.

Motivation & Objective

• To address the limitations of conventional machine learning models in emotion detection, which often ignore sequential and contextual information in text.
• To improve emotion detection performance by focusing on learning more meaningful latent representations of text through deep recurrent architectures.
• To evaluate whether capturing contextual and sequential dependencies enhances model generalization across low-resource and noisy emotion categories.
• To compare the impact of different pre-trained word embeddings (ConceptNet Numberbatch and fastText) on model performance in emotion classification.
• To validate the model’s robustness on a completely unseen, crowd-sourced dataset with non-standard labeling.

Proposed method

• The model employs a bidirectional gated recurrent unit (GRU) network to encode sequential dependencies in text, preserving context from both past and future tokens.
• A max-pooling layer is applied to extract the most salient features from the GRU’s hidden states, while an average-pooling layer captures the overall text representation.
• The model uses pre-trained word embeddings—ConceptNet Numberbatch and fastText—as input representations to enrich semantic and affective features.
• The final classification head uses a fully connected layer with softmax activation to predict discrete emotions across seven categories: joy, sadness, anger, love, fear, thankfulness, and surprise.
• The architecture is trained end-to-end using cross-entropy loss, with early stopping based on validation F1-score to prevent overfitting.
• The model is evaluated on two datasets: a hashtag-annotated tweet set and the noisy CrowdFlower dataset, with F1-measure as the primary metric.

Experimental results

Research questions

• RQ1 Can a bidirectional GRU-based model significantly outperform conventional machine learning methods in emotion detection by leveraging sequential and contextual information?
• RQ2 To what extent does learning a more informative latent representation improve performance across both high- and low-frequency emotion categories?
• RQ3 How do different pre-trained word embeddings (ConceptNet Numberbatch vs. fastText) affect the model’s performance and robustness?
• RQ4 Can the model generalize effectively to a completely unseen, noisy dataset with non-standard labeling, such as the CrowdFlower dataset?
• RQ5 Does the proposed architecture reduce variance in performance across different emotion classes compared to baseline methods?

Key findings

• The proposed bidirectional GRU model achieves an average 26.8-point increase in F1-measure over Wang et al.’s baseline on the primary dataset, with gains ranging from 10.0 points (joy) to 61.7 points (surprise).
• On the CrowdFlower dataset, the model shows a 38.6-point average F1-measure improvement over the baseline, with the highest gain of 57 points for the surprise emotion.
• The model demonstrates significantly reduced variance in performance across emotion categories compared to the baseline, indicating more stable and generalizable predictions.
• There is no meaningful performance difference between using ConceptNet Numberbatch and fastText embeddings, suggesting robustness to embedding choice.
• The model achieves high performance even on low-resource emotions like fear and surprise, indicating effective learning of subtle emotional cues.
• The model generalizes exceptionally well to a new, noisier dataset with non-standard labels, confirming its robustness and transferability.