[Paper Review] TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents

Thomas Wolf, Victor Sanh+2 more|arXiv (Cornell University)|Jan 23, 2019

Topic Modeling18 references282 citations

TL;DR

TransferTransfo fine-tunes a 12-layer decoder-only Transformer using multi-task learning (next-utterance classification + language modeling) on persona-chat, achieving state-of-the-art results on ConvAI2 metrics.

ABSTRACT

We introduce a new approach to generative data-driven dialogue systems (e.g. chatbots) called TransferTransfo which is a combination of a Transfer learning based training scheme and a high-capacity Transformer model. Fine-tuning is performed by using a multi-task objective which combines several unsupervised prediction tasks. The resulting fine-tuned model shows strong improvements over the current state-of-the-art end-to-end conversational models like memory augmented seq2seq and information-retrieval models. On the privately held PERSONA-CHAT dataset of the Conversational Intelligence Challenge 2, this approach obtains a new state-of-the-art, with respective perplexity, Hits@1 and F1 metrics of 16.28 (45 % absolute improvement), 80.7 (46 % absolute improvement) and 19.5 (20 % absolute improvement).

Motivation & Objective

• Motivate improvements in open-domain chatbots by addressing inconsistencies, lack of long-term memory, and generic responses.
• Leverage large pre-trained language models to enhance generative dialogue performance.
• Develop a fine-tuning scheme that jointly optimizes next-utterance prediction and language modeling on a persona-conditioned dialogue dataset.

Proposed method

• Use a 12-layer decoder-only Transformer with masked self-attention (768 hidden units, 12 heads).
• Pre-train on BooksCorpus with document-level sequences to capture long-range dependencies.
• Fine-tune with multi-task learning combining next-utterance classification and language modeling losses.
• Input representation concatenates persona sentences with dialog history and adds dialog-state embeddings indicating persona/ speaker roles.
• Promote invariance to persona sentence order by reusing positional embeddings and potentially shuffling persona sentences during training.
• Decode with beam search (beam size 4) plus sampling, applying n-gram filtering to avoid copying from persona sentences; final ranking combines utterance probability and next-utterance classification score.

Experimental results

Research questions

• RQ1 Can transfer learning from large language models improve the quality of open-domain conversational agents?
• RQ2 Does a multi-task fine-tuning setup (next-utterance classification + language modeling) yield better persona-consistent and fluent responses than traditional baselines?
• RQ3 How does the TransferTransfo perform on the ConvAI2 persona-chat dataset compared to end-to-end seq2seq and memory/retrieval baselines?

Key findings

• TransferTransfo achieves substantial improvements over ConvAI2 baselines on the persona-chat dataset in both validation and private test sets.
• Perplexity (PPL) improves by 45% absolute on the private test set and 51.0% on the validation set compared to certain baselines (exact figures reported in table).
• Hits@1 improves by approximately 46% absolute on the private test set and around 35% on the validation set.
• F1 scores improve by about 20% absolute on the private test set and around 13% on the validation set.
• The model achieves a PPL of 17.51 on eval and 16.28 on test, with Hits@1 of 82.1 (eval) and 80.7 (test), and F1 of 19.09 (eval) and 19.5 (test).
• Fine-tuning required roughly 10 hours on four K80 GPUs for 200,000 steps.