[Paper Review] Training Neural Response Selection for Task-Oriented Dialogue Systems

Matthew Henderson, Ivan Vulić+8 more|arXiv (Cornell University)|Jun 4, 2019

Topic Modeling70 references55 citations

TL;DR

A two-step training regime pretrains a neural response selection model on large Reddit data and then fine-tunes it for six target task-oriented dialogue domains, enabling effective adaptation in low-data settings.

ABSTRACT

Despite their popularity in the chatbot literature, retrieval-based models have had modest impact on task-oriented dialogue systems, with the main obstacle to their application being the low-data regime of most task-oriented dialogue tasks. Inspired by the recent success of pretraining in language modelling, we propose an effective method for deploying response selection in task-oriented dialogue. To train response selection models for task-oriented dialogue tasks, we propose a novel method which: 1) pretrains the response selection model on large general-domain conversational corpora; and then 2) fine-tunes the pretrained model for the target dialogue domain, relying only on the small in-domain dataset to capture the nuances of the given dialogue domain. Our evaluation on six diverse application domains, ranging from e-commerce to banking, demonstrates the effectiveness of the proposed training method.

Motivation & Objective

• Motivate response selection as an effective alternative to building task-specific ontologies and policies for task-oriented dialogue.
• Develop a lightweight two-step training procedure to overcome data scarcity in target domains.
• Evaluate pretrained-response-selection across diverse domains to assess portability and adaptation benefits.
• Identify encoder architectures and fine-tuning strategies that maximize performance after pretraining.

Proposed method

• Pretrain a response selection encoder on a large general-domain Reddit dataset with a transformer-like input-response encoder and scaled cosine similarity scoring.
• Represent input and response as 320-dimensional embeddings derived from unigram/bigram features processed through self-attention and projection layers, followed by a 512-dimensional final representation.
• Train with a batch-based max-margin objective that contrasts true input-response pairs against in-batch negatives, enabling efficient retrieval and potential approximate NN search.
• Fine-tune the pretrained model on six in-domain datasets (open-domain like OpenSubtitles and domain-specific like banking) using two strategies: ft-direct (continue training only on in-domain data) and ft-mixed (in-batch mixing of in-domain and Reddit data to retain general-domain knowledge).
• Compare against baselines including target-only training, traditional IR methods (tf-idf, BM25), and off-the-shelf pretrained encoders (use, elmo, bert variants) to assess the value of the two-step approach.

Experimental results

Research questions

• RQ1 Q1: Which encoder architecture and pretraining setup best model Reddit data for response selection?
• RQ2 Q2: How can a pretrained general-domain response selection model be efficiently adapted to diverse target dialogue domains?
• RQ3 Q3: Do lightweight fine-tuning strategies (ft-direct vs ft-mixed) outperform direct application of pretrained models and target-only training?
• RQ4 Q4: How does the pretrained+fine-tuned approach compare to strong vector-based baselines and standard IR baselines across multiple domains?

Key findings

• Pretraining on a large Reddit corpus yields superior input-response encodings over several baselines, with transformer-like architectures and self-attention providing gains.
• Fine-tuning with in-domain data substantially improves performance across all six target domains, especially when data are scarce.
• Two fine-tuning strategies (ft-direct and ft-mixed) outperform direct Reddit application and target-only training, with ft-direct offering strong gains when in-domain data are ample, while ft-mixed helps preserve general-domain capabilities.
• The two-step approach consistently beats traditional IR baselines (tf-idf, BM25) and off-the-shelf sentence encoders, underscoring the importance of task-specific adaptation for response selection.
• ft-mixed maintains Reddit performance while adapting to in-domain tasks, reducing degradation on the general-domain Reddit test data as in-domain data increase.