[Paper Review] Towards Conversational Recommendation over Multi-Type Dialogs

Zeming Liu, Haifeng Wang+4 more|arXiv (Cornell University)|May 8, 2020

Topic Modeling31 references19 citations

TL;DR

This paper introduces conversational recommendation over multi-type dialogs, where bots proactively transition from non-recommendation dialogs (e.g., QA, chitchat) to recommendation dialogs using user interests and feedback. The authors propose the MGCG framework with goal planning and goal-guided response generation, and release the DuRecDial dataset (10k dialogs, 156k utterances) to enable systematic study, achieving state-of-the-art results in goal completion and coherence with retrieval- and generation-based models.

ABSTRACT

We propose a new task of conversational recommendation over multi-type dialogs, where the bots can proactively and naturally lead a conversation from a non-recommendation dialog (e.g., QA) to a recommendation dialog, taking into account user's interests and feedback. To facilitate the study of this task, we create a human-to-human Chinese dialog dataset \emph{DuRecDial} (about 10k dialogs, 156k utterances), which contains multiple sequential dialogs for every pair of a recommendation seeker (user) and a recommender (bot). In each dialog, the recommender proactively leads a multi-type dialog to approach recommendation targets and then makes multiple recommendations with rich interaction behavior. This dataset allows us to systematically investigate different parts of the overall problem, e.g., how to naturally lead a dialog, how to interact with users for recommendation. Finally we establish baseline results on DuRecDial for future studies. Dataset and codes are publicly available at https://github.com/PaddlePaddle/models/tree/develop/PaddleNLP/Research/ACL2020-DuRecDial.

Motivation & Objective

• To address the lack of research on proactive, natural transitions from non-recommendation to recommendation dialogs in multi-type human-bot interactions.
• To study how bots can proactively guide conversations through multiple dialog types (e.g., QA, chitchat, task, recommendation) toward personalized recommendations.
• To model user interests and feedback for effective goal planning and topic transition in multi-type dialog flows.
• To create a large-scale, human-annotated, multi-domain dialog dataset supporting diverse dialog types and interaction patterns for conversational recommendation.

Proposed method

• Propose a multi-goal driven conversation generation (MGCG) framework with a goal planning module that determines long-term recommendation targets and short-term goals based on user interests and topic naturalness.
• Design a goal-guided responding module that generates contextually appropriate, knowledge-grounded responses for each dialog type (e.g., QA, chitchat, task, recommendation).
• Integrate knowledge graph embeddings and user profile information to enhance response informativeness and personalization.
• Use both retrieval-based and generation-based response selection strategies, with the latter trained end-to-end using sequence-to-sequence modeling.
• Implement a mixed-goal driven dialog policy that manages flow across different dialog types while maintaining coherence and proactivity.
• Evaluate the framework on the newly released DuRecDial dataset using automatic and human evaluation metrics, including goal success rate, coherence, and proactivity.

Experimental results

Research questions

• RQ1 How can a bot proactively and naturally transition from non-recommendation dialogs (e.g., QA, chitchat) to recommendation dialogs in a multi-type dialog setting?
• RQ2 What role does user interest modeling and feedback play in planning effective goal sequences for conversational recommendation?
• RQ3 How can knowledge grounding and multi-turn coherence be maintained across diverse dialog types in a single conversation?
• RQ4 To what extent can retrieval-based and generation-based response models differ in performance on goal completion and naturalness in multi-type dialog systems?
• RQ5 What are the key challenges in achieving high goal success rate and coherence when managing multiple dialog types simultaneously?

Key findings

• The MGCG framework with retrieval-based and generation-based models significantly outperforms the S2S baseline in appropriateness, informativeness, goal success rate, and coherence.
• The retrieval-based model achieves higher fluency due to selecting from real human utterances, but underperforms in goal completion and informativeness due to limited candidate diversity.
• Goal success rate strongly correlates with knowledge usage: models using more knowledge achieve higher goal completion, especially in challenging dialog types like QA and recommendation.
• Chitchat goals are easier to complete than QA or recommendation goals, indicating higher complexity in knowledge-intensive dialog types.
• The generation-based MGCG model achieves 174 completed goals out of 272 total predicted goals (64% success rate), while the retrieval-based model achieves 158 out of 288 (55% success rate), showing improved performance over S2S.
• Human evaluation confirms that the MGCG models generate more proactive, coherent, and goal-oriented responses than the S2S baseline, with higher scores in appropriateness and informativeness.