[Paper Review] Modeling Multi-turn Conversation with Deep Utterance Aggregation

Zhuosheng Zhang, Jiangtong Li+3 more|arXiv (Cornell University)|Jun 24, 2018

Topic Modeling41 references92 citations

TL;DR

The paper proposes a Deep Utterance Aggregation (DUA) model for retrieval-based multi-turn dialogue, using turns-aware aggregation and self-matching attention to form fine-grained context representations, achieving state-of-the-art results on Ubuntu, Douban, and a new E-commerce Dialogue Corpus (ECD).

ABSTRACT

Multi-turn conversation understanding is a major challenge for building intelligent dialogue systems. This work focuses on retrieval-based response matching for multi-turn conversation whose related work simply concatenates the conversation utterances, ignoring the interactions among previous utterances for context modeling. In this paper, we formulate previous utterances into context using a proposed deep utterance aggregation model to form a fine-grained context representation. In detail, a self-matching attention is first introduced to route the vital information in each utterance. Then the model matches a response with each refined utterance and the final matching score is obtained after attentive turns aggregation. Experimental results show our model outperforms the state-of-the-art methods on three multi-turn conversation benchmarks, including a newly introduced e-commerce dialogue corpus.

Motivation & Objective

• Motivate improved context modeling for multi-turn retrieval-based dialogue by moving beyond naive concatenation of prior utterances.
• Develop a turns-aware aggregation mechanism to fuse last utterance with preceding context.
• Highlight salient information within each utterance via self-matching attention.
• Match responses to refined utterances at word and utterance levels and aggregate results for final scoring.
• Evaluate the approach on multiple benchmarks including a newly released e-commerce dialogue corpus and compare against strong baselines.

Proposed method

• Represent each utterance and the response with word-level GRU encoders.
• Apply a turns-aware aggregation that fuses each prior utterance with the last utterance (concatenation chosen as the aggregation method).
• Use self-matching attention to filter redundant information within the fused utterances sequence.
• Construct word- and utterance-level matching matrices between each utterance and the response, and encode them with CNNs to obtain matching vectors.
• Process the sequence of matching vectors with a gated recurrent unit (GRU) in chronological order and produce a final score via attention over the GRU outputs.
• Train the model with cross-entropy loss.

Experimental results

Research questions

• RQ1 Can turns-aware aggregation improve context representation for multi-turn retrieval-based dialogue beyond simple concatenation?
• RQ2 Does self-matching attention effectively distill salient information within utterances to improve response matching?
• RQ3 How does the proposed Deep Utterance Aggregation (DUA) perform relative to state-of-the-art baselines on multiple multi-turn dialogue benchmarks, including an English Ubuntu dataset, a Chinese Douban dataset, and a newly released e-commerce corpus?
• RQ4 What insights can be drawn from ablation analyses about the importance of context fusion and matching attention flow?

Key findings

• DUA outperforms existing models on three multi-turn dialogue benchmarks (Ubuntu, Douban, and ECD).
• The model achieves notable improvements over the previous state-of-the-art on the ECD dataset, including a 4.8% gain in R10@1 over SMN.
• Ablation studies show that both Context Fusion (turns-aware aggregation) and Matching Attention Flow are important, with the largest drop when Matching Attention Flow is removed.
• A qualitative analysis demonstrates that self-matching attention effectively identifies and concentrates on crucial parts of utterances and responses to guide matching.
• The authors release the first public e-commerce dialogue corpus (ECD) to the research community, enabling broader evaluation in service-oriented conversations.