[Paper Review] DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation

Yizhe Zhang, Siqi Sun+7 more|arXiv (Cornell University)|Nov 1, 2019

Topic Modeling28 references103 citations

TL;DR

DialoGPT is a large-scale, open-domain conversational response generation model built on GPT-2, trained on 147M Reddit dialogue exchanges, achieving near-human performance in single-turn dialogues and releasing pre-trained models and training pipeline for research.

ABSTRACT

We present a large, tunable neural conversational response generation model, DialoGPT (dialogue generative pre-trained transformer). Trained on 147M conversation-like exchanges extracted from Reddit comment chains over a period spanning from 2005 through 2017, DialoGPT extends the Hugging Face PyTorch transformer to attain a performance close to human both in terms of automatic and human evaluation in single-turn dialogue settings. We show that conversational systems that leverage DialoGPT generate more relevant, contentful and context-consistent responses than strong baseline systems. The pre-trained model and training pipeline are publicly released to facilitate research into neural response generation and the development of more intelligent open-domain dialogue systems.

Motivation & Objective

• Motivate open-domain dialogue generation with a large, pre-trained transformer model.
• Show that dialogue-specific pre-training on large conversational data improves response quality.
• Demonstrate state-of-the-art automatic and human evaluation performance on DSTC-7 and multi-reference Reddit datasets.
• Provide an open-source training pipeline and pre-trained models to facilitate further research.

Proposed method

• Extend GPT-2 architecture to model multi-turn dialogues by concatenating dialogue turns into a long text and maximizing the conditional probability P(T|S).
• Train three model sizes (117M, 345M, 762M parameters) on 147M Reddit dialogues with a 50,257 vocab using byte-pair encoding.
• Apply a maximum mutual information (MMI) re-ranking step using a backward model to prefer informative and less bland responses.
• Experiment with generation settings including top-K sampling and beam search to evaluate diversity and quality.
• Evaluate using automatic metrics (NIST, BLEU, METEOR, Entropy, Dist) and human evaluation on DSTC-7 and a 6K Reddit multi-reference test set.

Experimental results

Research questions

• RQ1 Can a GPT-2-based model trained on large-scale dialogue data generate more relevant, contentful, and context-consistent open-domain responses than baselines?
• RQ2 Does larger model size and decoding strategies (beam search) improve automatic and human evaluation scores in open-domain dialogue generation?
• RQ3 Does backward-model re-ranking via mutual information improve response diversity and informativeness without sacrificing relevance?
• RQ4 How does fine-tuning from GPT-2 pre-training compare to training from scratch on Reddit data for dialogue generation?

Key findings

• DialoGPT-345M with beam search achieved the highest automatic scores across most metrics on the 6K Reddit multi-reference test set.
• Larger models (345M and 762M) outperform the 117M variant across automatic metrics and human evaluations.
• Beam search substantially improves BLEU and Dist-n metrics, with marginal gains in NIST and METEOR.
• MMI re-ranking increases diversity and information content (higher NIST, METEOR, Entropy, Dist) with a slight BLEU drop.
• Human evaluations show strong preference for DialoGPT variants over a production-like baseline (PersonalityChat), with some cases where MMI variants approach or exceed human preferences in certain metrics.