[Paper Review] Dialogue Act Classification with Context-Aware Self-Attention

Vipul Raheja, Joel Tetreault|arXiv (Cornell University)|Apr 4, 2019

Speech and dialogue systems29 references60 citations

TL;DR

The paper presents a hierarchical model using context-aware self-attention for dialogue act classification, achieving state-of-the-art results on SwDA and competitive results on MRDA by integrating utterance-level representations with conversation-wide context and CRF decoding.

ABSTRACT

Recent work in Dialogue Act classification has treated the task as a sequence labeling problem using hierarchical deep neural networks. We build on this prior work by leveraging the effectiveness of a context-aware self-attention mechanism coupled with a hierarchical recurrent neural network. We conduct extensive evaluations on standard Dialogue Act classification datasets and show significant improvement over state-of-the-art results on the Switchboard Dialogue Act (SwDA) Corpus. We also investigate the impact of different utterance-level representation learning methods and show that our method is effective at capturing utterance-level semantic text representations while maintaining high accuracy.

Motivation & Objective

• Motivate improved dialogue act (DA) classification by leveraging conversation-level context.
• Develop a hierarchical model that combines utterance-level representations with context-aware self-attention.
• Incorporate previous utterance context via a conversation-level RNN and CRF decoder to model label dependencies.
• Evaluate the model on SwDA and MRDA datasets and analyze utterance representation learning methods.

Proposed method

• Use an utterance-level RNN to encode words within each utterance, combining GloVe, ELMo, and character-level CNN embeddings.
• Apply a context-aware self-attention mechanism that incorporates the previous conversation state to produce utterance representations.
• Construct a conversation-level bidirectional GRU over utterance embeddings, followed by a CRF layer to jointly decode the sequence of dialogue acts.
• Experiment with various utterance representation learning approaches (TF-IDF GloVe baseline, Skip-Thought, Paragraph Vectors, and jointly learned representations).
• Optionally compare attention variants (word-level attention, self-attention) and assess the impact of context on performance.

Experimental results

Research questions

• RQ1 Does context-aware self-attention improve utterance representations for DA classification compared to non-contextual representations?
• RQ2 How does incorporating conversation-level context (previous hidden state) affect classification accuracy on SwDA and MRDA?
• RQ3 What is the impact of different utterance representation learning methods on overall model performance?
• RQ4 How does CRF decoding influence sequence labeling performance in this hierarchical setup?

Key findings

• The proposed method achieves 82.9% accuracy on SwDA and 91.1% on MRDA, outperforming several baselines and many previous methods.
• Our method surpasses state-of-the-art on SwDA by 1.6% and comes within 0.6% on MRDA.
• Contextual information significantly boosts performance, especially when combined with CRF decoding.
• Context-aware self-attention yields better utterance representations than non-contextual or baseline representations.
• Character-level embeddings and rich utterance representations (GloVe/ELMo) contribute to improvements in both datasets.