[Paper Review] Emotion Detection on TV Show Transcripts with Sequence-based Convolutional Neural Networks

Sayyed M. Zahiri, Jinho D. Choi|arXiv (Cornell University)|Aug 14, 2017

Sentiment Analysis and Opinion Mining31 references110 citations

TL;DR

The paper introduces a new Friends transcript emotion corpus with seven fine-grained emotions and proposes sequence-based CNN models with attention to capture dialogue dynamics, achieving improvements over baseline CNNs.

ABSTRACT

While there have been significant advances in detecting emotions from speech and image recognition, emotion detection on text is still under-explored and remained as an active research field. This paper introduces a corpus for text-based emotion detection on multiparty dialogue as well as deep neural models that outperform the existing approaches for document classification. We first present a new corpus that provides annotation of seven emotions on consecutive utterances in dialogues extracted from the show, Friends. We then suggest four types of sequence-based convolutional neural network models with attention that leverage the sequence information encapsulated in dialogue. Our best model shows the accuracies of 37.9% and 54% for fine- and coarse-grained emotions, respectively. Given the difficulty of this task, this is promising.

Motivation & Objective

• Create a large, fine-grained text emotion corpus for multiparty dialogue from TV show transcripts.
• Develop sequence-based CNN models that leverage utterance sequences with attention for emotion classification.
• Provide comprehensive corpus analytics and compare SCNN models to baselines.
• Demonstrate that attention-enhanced SCNN models outperform standard CNNs on this task.

Proposed method

• Annotate Friends transcripts with seven emotions (sad, mad, scared, powerful, peaceful, joyful, neutral) across consecutive utterances.
• Propose sequence-based CNN models (SCNN) that fuse current utterance features with previous utterances via concatenation and two variants (SCNN c and SCNN v).
• Incorporate attention mechanisms (SCNN c^a and SCNN v^a) to weigh past utterances when predicting the current emotion.
• Train word embeddings (Word2Vec, 200-d) and use CNN-based feature extractors with 1-D convolutions for sequence fusion.
• Compare SCNN variants against CNN and RNN-CNN baselines on a train/dev/test split of episodes.

Experimental results

Research questions

• RQ1 Can a text-only, sequence-aware neural model accurately detect emotions in multiparty dialogue from TV transcripts?
• RQ2 Do sequence-based CNNs with attention outperform standard CNNs and RNN-based models on this task?
• RQ3 How does using consecutive utterances influence emotion detection performance in dialogue data?

Key findings

• The best model (SCNN_c^a) achieves 37.90% accuracy and 54.00% macro F1 on 7-classEmotion classification and 39.25% macro F1 on 3-class (coarse) evaluation.
• SCNN variants outperform the base CNN and RNN-CNN baselines on evaluation metrics.
• Attention mechanisms provide gains over non-attentive SCNN models, with SCNN_c^a yielding the highest reported scores.
• Using three to five preceding utterances yields the best performance, with diminishing returns for longer histories in longer scenes.