[Paper Review] MaskGAN: Better Text Generation via Filling in the______

William Fedus, Ian Goodfellow+1 more|arXiv (Cornell University)|Jan 23, 2018

Topic Modeling6 references256 citations

TL;DR

MaskGAN introduces an actor-critic conditional GAN that fills in missing text conditioned on context, improving sample quality over standard ML-trained baselines in PTB and IMDB datasets.

ABSTRACT

Neural text generation models are often autoregressive language models or seq2seq models. These models generate text by sampling words sequentially, with each word conditioned on the previous word, and are state-of-the-art for several machine translation and summarization benchmarks. These benchmarks are often defined by validation perplexity even though this is not a direct measure of the quality of the generated text. Additionally, these models are typically trained via maxi- mum likelihood and teacher forcing. These methods are well-suited to optimizing perplexity but can result in poor sample quality since generating text requires conditioning on sequences of words that may have never been observed at training time. We propose to improve sample quality using Generative Adversarial Networks (GANs), which explicitly train the generator to produce high quality samples and have shown a lot of success in image generation. GANs were originally designed to output differentiable values, so discrete language generation is challenging for them. We claim that validation perplexity alone is not indicative of the quality of text generated by a model. We introduce an actor-critic conditional GAN that fills in missing text conditioned on the surrounding context. We show qualitatively and quantitatively, evidence that this produces more realistic conditional and unconditional text samples compared to a maximum likelihood trained model.

Motivation & Objective

• Motivation to improve sample quality beyond validation perplexity for autoregressive and seq2seq text generators.
• Propose a text in-filling task where missing tokens are filled conditioned on surrounding context.
• Train a generator with reinforcement learning using an actor-critic framework guided by a discriminator.
• Mitigate training instability and mode collapse common in text GANs through the in-filling setup and critic assistance.

Proposed method

• Use a seq2seq generator that fills in masked tokens conditioned on past and future context.
• Train a discriminator that evaluates each filled token given the original context to provide token-level rewards.
• Adopt an actor-critic reinforcement learning objective where the critic provides a baseline to reduce gradient variance.
• Define rewards as the log-probabilities from the discriminator and optimize with policy gradients.
• Pretrain with maximum likelihood language models before adversarial fine-tuning on the in-filling task.
• Optionally employ curriculum-like adjustments for long sequences and large vocabularies.

Experimental results

Research questions

• RQ1 Can a text in-filling task with a token-level discriminator provide better sample quality than conventional ML-trained models?
• RQ2 Does an actor-critic training setup reduce gradient variance and stabilize GAN-based text generation?
• RQ3 How does MaskGAN perform in conditional and unconditional text generation on PTB and IMDB compared to MaskMLE?
• RQ4 What evidence from human evaluation supports improvements in realism of generated text?
• RQ5 How do BLEU-inspired and n-gram diversity metrics reflect mode coverage and sample quality in MaskGAN?

Key findings

• MaskGAN produces more realistic conditional and unconditional text samples than a maximum likelihood trained model according to qualitative and some quantitative measures.
• Perplexity-based measures on generated samples do not always align with sample quality; MaskGAN shows lower perplexity on generated samples than the baseline in some setups.
• Human evaluation favors MaskGAN samples over MaskMLE on the IMDB dataset across grammaticality, topicality, and overall quality.
• MaskGAN exhibits some mode collapse as indicated by reduced unique quadgrams, but overall generated samples remain diverse across models.
• The prior training and inclusion of a critic significantly reduce gradient variance and improve training stability.