[Paper Review] Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions

Stephen Roller, Y-Lan Boureau+14 more|arXiv (Cornell University)|Jun 22, 2020

Topic ModelingComputer Science149 references42 citations

TL;DR

This position paper surveys the desired traits, current progress, and open problems in building open-domain conversational agents, emphasizing continual learning, engaging content, and well-behaved behavior, with a bias toward the authors’ group work.

ABSTRACT

We present our view of what is necessary to build an engaging open-domain conversational agent: covering the qualities of such an agent, the pieces of the puzzle that have been built so far, and the gaping holes we have not filled yet. We present a biased view, focusing on work done by our own group, while citing related work in each area. In particular, we discuss in detail the properties of continual learning, providing engaging content, and being well-behaved -- and how to measure success in providing them. We end with a discussion of our experience and learnings, and our recommendations to the community.

Motivation & Objective

• Define the long-term goal of a superhuman open-domain conversational agent and distinguish it from a Turing-style test.
• Identify and delineate the key traits (continual learning, engaging content, well-behaved) required for engaging open-domain chatbots.
• Summarize the main approaches and experiments from the authors’ group on end-to-end dialogue modeling and knowledge integration.
• Discuss mechanisms for evaluating progress and the practical challenges of deploying open-domain agents in the wild.

Proposed method

• Review and synthesize recent work on end-to-end dialogue models and their components (memory, knowledge grounding, generation, and evaluation).
• Describe data source strategies for continual online training and the use of static versus dynamic benchmarks.
• Discuss feedback learning from interaction, including self-feeding and implicit quality signals, and their reinforcement learning implications.
• Explore knowledge updating through retrieval and dynamic sources, and compare generative versus retrieval-based approaches.

Experimental results

Research questions

• RQ1 What characteristics define a superior open-domain conversational agent and how can they be measured?
• RQ2 What are the main challenges and gaps in continual learning, knowledge updating, and memory for long-term conversations?
• RQ3 How can agents learn from interaction signals to improve dialogue quality without explicit supervised ratings?
• RQ4 How should open-domain models balance engagement, correctness, and safety while staying current with topics?

Key findings

• Continual learning requires online training, learning signals from interaction, and dynamic knowledge updating, with open problems in forgetting and data distribution shift.
• Learning from conversation feedback can improve performance, but distinguishing conversation-specific signals from external factors remains challenging.
• Generative models can surpass retrieval models with large pre-training and decoding improvements, yet still face issues with consistency and diversity.
• Techniques like controlling generation (specificity, repetition, question-asking) and unlikelihood training show promise for more engaging and less degenerate outputs.
• Memory, reasoning, and commonsense remain major gaps; current approaches often rely on short-term history and retrieval rather than long-term, integrated knowledge.