[Paper Review] Decision support with text-based emotion recognition: Deep learning for affective computing

Bernhard Kratzwald, Suzana Ilić+3 more|arXiv (Cornell University)|Mar 16, 2018

Sentiment Analysis and Opinion MiningComputer Science65 references18 citations

TL;DR

This paper proposes a deep learning approach using recurrent neural networks (RNNs) with transfer learning to improve text-based emotion recognition for affective computing in decision support. It achieves up to 21% higher F1-score and 6% lower forecast error than traditional methods, demonstrating superior performance in classifying emotions and predicting affective intensity.

ABSTRACT

Emotions widely affect the decision-making of humans and, hence, affective computing takes emotional states into account with the goal of tailoring decision support to individuals. However, the accurate recognition of emotions within narrative materials presents a challenging undertaking due to the complexity and ambiguity of language. Even though deep learning has evolved as the state-of-the-art in various tasks from text mining, its benefits with regard to affective computing are not yet understood. We thus propose the following innovations: (1) we adapt recurrent neural networks from the field of deep learning to affective computing. (2) We extend these networks for predicting the score of different affective dimensions. (3) We implement transfer learning for pre-training word embeddings. Analyzing the results, we find that deep learning consistently outperforms traditional machine learning with improvements of up to 21% in F1-score when labeling emotions and 6% in forecast errors when rating the intensity of emotions. Altogether, the findings have considerable implications for the use of affective computing in providing decision support.

Motivation & Objective

• To address the challenge of accurately recognizing emotions in narrative text, which is complex and ambiguous due to language nuances.
• To adapt deep learning techniques, particularly recurrent neural networks, to the domain of affective computing for improved emotion detection.
• To extend RNNs to predict scores across multiple affective dimensions, such as emotion intensity and valence.
• To implement transfer learning for pre-training word embeddings to enhance model generalization and performance.
• To evaluate the effectiveness of deep learning in comparison to traditional machine learning methods in affective computing tasks.

Proposed method

• Recurrent neural networks (RNNs) are adapted from deep learning to model sequential dependencies in textual emotion data.
• The RNN architecture is extended to predict continuous scores across multiple affective dimensions, enabling fine-grained emotion intensity estimation.
• Transfer learning is applied to pre-train word embeddings using large-scale text corpora, improving representation learning for emotion classification.
• The model is trained end-to-end on narrative text to map input sequences to affective dimension scores.
• Performance is evaluated using standard metrics such as F1-score for emotion labeling and forecast error for intensity prediction.
• Comparative experiments are conducted between deep learning models and traditional machine learning baselines to assess relative gains.

Experimental results

Research questions

• RQ1 Can recurrent neural networks effectively recognize emotions in narrative text, outperforming traditional machine learning methods?
• RQ2 To what extent does transfer learning for word embedding pre-training improve emotion recognition performance?
• RQ3 How well can deep learning models predict continuous scores across multiple affective dimensions, such as intensity and valence?
• RQ4 What improvements in F1-score and forecast error are achieved by deep learning compared to traditional approaches in affective computing?

Key findings

• Deep learning models achieve up to 21% higher F1-score in emotion classification compared to traditional machine learning methods.
• The proposed approach reduces forecast error by 6% when predicting the intensity of emotions, indicating improved regression performance.
• Transfer learning for word embedding pre-training contributes significantly to enhanced model generalization and accuracy.
• The extended RNN architecture successfully captures sequential patterns in text to predict multi-dimensional affective states.
• Consistent performance gains are observed across both emotion labeling and intensity prediction tasks, validating the approach's robustness.