[Paper Review] Proactive Human-Machine Conversation with Explicit Conversation Goals

Wenquan Wu, Zhen Guo+5 more|arXiv (Cornell University)|Jun 13, 2019

Topic Modeling20 references41 citations

TL;DR

The paper introduces a proactive dialogue setting where a leader, aided by a knowledge graph, guides a conversation toward explicit topics, and provides DuConv, a large dataset for knowledge-driven proactive conversations; it evaluates retrieval- and generation-based models that plan over the knowledge graph.

ABSTRACT

Though great progress has been made for human-machine conversation, current dialogue system is still in its infancy: it usually converses passively and utters words more as a matter of response, rather than on its own initiatives. In this paper, we take a radical step towards building a human-like conversational agent: endowing it with the ability of proactively leading the conversation (introducing a new topic or maintaining the current topic). To facilitate the development of such conversation systems, we create a new dataset named DuConv where one acts as a conversation leader and the other acts as the follower. The leader is provided with a knowledge graph and asked to sequentially change the discussion topics, following the given conversation goal, and meanwhile keep the dialogue as natural and engaging as possible. DuConv enables a very challenging task as the model needs to both understand dialogue and plan over the given knowledge graph. We establish baseline results on this dataset (about 270K utterances and 30k dialogues) using several state-of-the-art models. Experimental results show that dialogue models that plan over the knowledge graph can make full use of related knowledge to generate more diverse multi-turn conversations. The baseline systems along with the dataset are publicly available

Motivation & Objective

• Motivate building human-like agents that proactively lead conversations rather than just respond.
• Propose a knowledge-grounded proactive dialogue task with explicit goals over a knowledge graph.
• Construct and release a large-scale dataset (DuConv) combining factoid and non-factoid knowledge.
• Develop and benchmark knowledge-aware retrieval and generation models for proactive dialogue.
• Analyze how planning over a knowledge graph improves diversity and goal achievement in multi-turn conversations.

Proposed method

• Construct a knowledge graph by linking factoid and non-factoid movie knowledge, including associated triplets.
• Define conversation goals as knowledge paths [start -> topic_a -> topic_b] and collect ~30k dialogues with a leader and a follower.
• Propose a knowledge-aware retrieval-based model with context, knowledge, and goal integration for response ranking.
• Propose a knowledge-aware generation-based model that learns prior/posterior knowledge distributions and uses KL divergence to align them, with a knowledge-guided decoder.
• Train and evaluate models with automatic metrics (Hits@1/3, PPL, F1/BLEU, DISTINCT) and human judgments (fluency, coherence, informativeness, proactivity, goal completion).
• Release code and data publicly to enable benchmarking on knowledge-driven proactive dialogue.

Experimental results

Research questions

• RQ1 Can a dialogue system proactively steer conversations by planning over a knowledge graph with an explicit goal?
• RQ2 Does incorporating knowledge graphs and goal-directed planning improve diversity and informativeness in multi-turn dialogues?
• RQ3 How do retrieval-based and generation-based approaches compare in knowledge-grounded proactive dialogue settings?
• RQ4 What is the impact of normalizing topic labels (topic_a/topic_b) on generalization and performance?

Key findings

• Knowledge-aware models outperform baselines that do not use knowledge on overall measurements.
• Generation-based models achieve higher turn-level informativeness and proactivity and better dialogue-level goal completion and coherence than retrieval-based or Seq2Seq baselines.
• Incorporating prior/posterior knowledge distributions with KL divergence improves knowledge utilization and goal achievement.
• Normalizing topic labels (topic_a/topic_b) significantly enhances performance across models.
• Knowledge-enabled approaches yield more diverse and topic-appropriate responses, though full multi-turn goal completion remains challenging in some cases.