[Paper Review] Language Generation with Recurrent Generative Adversarial Networks without Pre-training

Ofir Press, Amir Bar+3 more|arXiv (Cornell University)|Jun 5, 2017

Generative Adversarial Networks and Image SynthesisComputer Science19 references90 citations

TL;DR

The paper trains an RNN-based GAN for language generation from scratch using curriculum learning, variable-length training, and teacher helping to produce coherent text without pre-training. It shows improvements over a CNN baseline and generalizes to longer sequences.

ABSTRACT

Generative Adversarial Networks (GANs) have shown great promise recently in image generation. Training GANs for language generation has proven to be more difficult, because of the non-differentiable nature of generating text with recurrent neural networks. Consequently, past work has either resorted to pre-training with maximum-likelihood or used convolutional networks for generation. In this work, we show that recurrent neural networks can be trained to generate text with GANs from scratch using curriculum learning, by slowly teaching the model to generate sequences of increasing and variable length. We empirically show that our approach vastly improves the quality of generated sequences compared to a convolutional baseline.

Motivation & Objective

• Motivate learning language generation with GANs without pre-training to overcome exposure bias and strict ML loss.
• Demonstrate that an RNN generator and discriminator can be trained end-to-end under an Improved Wasserstein GAN objective.
• Show that curriculum learning with increasing sequence lengths aids stable training and quality of generated text.
• Evaluate how extensions like Variable Length and Teacher Helping improve sequence quality and generalization.

Proposed method

• Use a GRU-based RNN for both generator and discriminator with continuous relaxation of inputs for differentiability.
• Train under the Improved Wasserstein GAN objective with a discriminator providing a scalar score.
• Employ curriculum learning to incrementally increase generated sequence length during training.
• Incorporate Variable Length training where sequences of multiple lengths are used within batches.
• Introduce Teacher Helping by conditioning generation on short ground truth prefixes to guide the final character.
• Experiment with noise drawn from N(0,10) and train the generator more frequently than the discriminator to stabilize learning.

Experimental results

Research questions

• RQ1 Can an RNN-based generator be trained with a GAN objective from scratch without pre-training to generate coherent text?
• RQ2 Do curriculum learning, variable-length training, and teacher helping improve the quality and coherence of GAN-generated text compared to CNN baselines?
• RQ3 Does the model generalize to sequences longer than those seen during training?
• RQ4 What is the impact of training dynamics (generator/discriminator update ratio, noise scale) on text generation quality?

Key findings

• An RNN-based GAN can learn to generate natural language from scratch using the Improved Wasserstein GAN objective.
• The combination of curriculum learning, variable length, and teacher helping yields the best results, outperforming the baseline CNN model on word n-gram similarity to held-out data.
• The CL+VL+TH model achieves 3.8% of word 4-grams from generated text also appearing in the held-out test set for length 64 sequences.
• Curriculum learning alone is not sufficient; variable length and teacher helping are important for performance gains.
• The model trained on sequences up to length 32 generalizes to sequences of length 64 with only a small degradation in performance.