[Paper Review] BiERU: Bidirectional Emotional Recurrent Unit for Conversational Sentiment Analysis

Wei Li, Wei Shao+2 more|arXiv (Cornell University)|May 31, 2020

Sentiment Analysis and Opinion Mining64 references40 citations

TL;DR

BiERU introduces a fast, compact party-ignorant framework with a Generalized Neural Tensor Block (GNTB) and a Two-Channel Feature Extractor (TFE) to model context in conversations for sentiment analysis; bidirectional variants BiERU-gc and BiERU-lc achieve state-of-the-art results with fewer parameters.

ABSTRACT

Sentiment analysis in conversations has gained increasing attention in recent years for the growing amount of applications it can serve, e.g., sentiment analysis, recommender systems, and human-robot interaction. The main difference between conversational sentiment analysis and single sentence sentiment analysis is the existence of context information which may influence the sentiment of an utterance in a dialogue. How to effectively encode contextual information in dialogues, however, remains a challenge. Existing approaches employ complicated deep learning structures to distinguish different parties in a conversation and then model the context information. In this paper, we propose a fast, compact and parameter-efficient party-ignorant framework named bidirectional emotional recurrent unit for conversational sentiment analysis. In our system, a generalized neural tensor block followed by a two-channel classifier is designed to perform context compositionality and sentiment classification, respectively. Extensive experiments on three standard datasets demonstrate that our model outperforms the state of the art in most cases.

Motivation & Objective

• Address the challenge of encoding contextual information in conversational sentiment analysis.
• Develop a fast, compact, and parameter-efficient party-ignorant model that handles multi-turn conversations.
• Introduce a mechanism for context compositionality that integrates context into utterance representations.
• Extract emotion features efficiently to improve sentiment classification and regression.
• Demonstrate state-of-the-art performance on standard benchmarks with fewer parameters.

Proposed method

• Propose a bidirectional ERU (BiERU) consisting of a Generalized Neural Tensor Block (GNTB) and a Two-channel Feature Extractor (TFE).
• GNTB models context compositionality by combining previous contextual utterances with the current utterance to produce a contextual utterance vector p_t.
• TFE uses an LSTM branch and a 1D-CNN branch to extract emotion features from p_t, producing e_t which is used for prediction.
• Implement two BiERU variants: BiERU-gc (global context) and BiERU-lc (local context) to capture different contextual information sources.
• Train the model with cross-entropy loss for classification or MSE loss for regression, using L2 regularization and Adam optimization.

Experimental results

Research questions

• RQ1 Can a compact, party-ignorant model effectively capture conversational context for sentiment analysis without explicit speaker separation?
• RQ2 Does context compositionality via GNTB improve utterance representations and downstream sentiment classification/regression compared to existing methods?
• RQ3 How do global vs. local contextual inputs affect performance across standard conversational sentiment benchmarks?
• RQ4 What is the parameter efficiency and training efficiency of BiERU relative to state-of-the-art party-dependent and party-ignorant baselines?

Key findings

• BiERU-lc achieves state-of-the-art results on IEMOCAP, MELD, and competitive performance on AVEC using textual modality.
• BiERU models outperform baselines like DialogueRNN, DialogueGCN, and AGHMN on several metrics with fewer parameters.
• BiERU-lc generally outperforms BiERU-gc, suggesting local contextual information yields better sentiment modeling.
• Ablation shows that combining GNTB and TFE yields substantial accuracy/F1 improvements over using either component alone.
• BiERU demonstrates strong parameter efficiency (e.g., ~0.5M params on IEMOCAP vs ~1M for DialogueRNN) and faster training times.