[Paper Review] Long Text Generation via Adversarial Training with Leaked Information

Jiaxian Guo, Sidi Lu+4 more|arXiv (Cornell University)|Sep 24, 2017

Topic Modeling28 references161 citations

TL;DR

LeakGAN introduces a hierarchical generator that leverages leaked discriminator features to guide long-text generation, improving both synthetic and real-data BLEU scores and human judgments.

ABSTRACT

Automatically generating coherent and semantically meaningful text has many applications in machine translation, dialogue systems, image captioning, etc. Recently, by combining with policy gradient, Generative Adversarial Nets (GAN) that use a discriminative model to guide the training of the generative model as a reinforcement learning policy has shown promising results in text generation. However, the scalar guiding signal is only available after the entire text has been generated and lacks intermediate information about text structure during the generative process. As such, it limits its success when the length of the generated text samples is long (more than 20 words). In this paper, we propose a new framework, called LeakGAN, to address the problem for long text generation. We allow the discriminative net to leak its own high-level extracted features to the generative net to further help the guidance. The generator incorporates such informative signals into all generation steps through an additional Manager module, which takes the extracted features of current generated words and outputs a latent vector to guide the Worker module for next-word generation. Our extensive experiments on synthetic data and various real-world tasks with Turing test demonstrate that LeakGAN is highly effective in long text generation and also improves the performance in short text generation scenarios. More importantly, without any supervision, LeakGAN would be able to implicitly learn sentence structures only through the interaction between Manager and Worker.

Motivation & Objective

• Address the difficulty of long-text generation under GAN by providing informative, step-by-step guidance during generation.
• Leverage discriminator-internal features as leaked information to guide the generator.
• Introduce a hierarchical generator (MANAGER and WORKER) to utilize leaked features effectively.
• Demonstrate improvements on synthetic data and real-world corpora, and analyze learned sentence structure without supervision.

Proposed method

• Discriminator Dφ provides high-level features f from its CNN-based feature extractor.
• A hierarchical generator Gθ with a MANAGER (LSTM) and WORKER (LSTM) uses leaked features ft to produce a goal gt and a goal embedding wt.
• The WORKER uses the current word xt and the goal embedding to generate the next word via a softmax over logits Ot · wt / α.
• TRAINING uses REINFORCE to optimize the WORKER; the MANAGER is trained to predict advantageous directions in the discriminator feature space (cosine similarity with ft+c − ft).
• Pre-training involves supervised stages for both MANAGER and WORKER, followed by alternating adversarial training between G and D; additional techniques include bootstrapped rescaled activation, interleaved training, and temperature control.

Experimental results

Research questions

• RQ1 Can leaked high-level discriminator features improve guidance for long-text generation in a GAN setting?
• RQ2 Does a hierarchical (manager-worker) generator better utilize leaked information to produce coherent long text?
• RQ3 How does LeakGAN perform on synthetic data and diverse real-world corpora (long, mid-length, short text)?
• RQ4 What insight into learned sentence structure can be gained from the manager-worker interaction?

Key findings

• LeakGAN achieves lower negative log-likelihood than baselines on synthetic data for longer sequences.
• LeakGAN yields higher BLEU scores than SeqGAN and RankGAN on EMNLP2017 WMT News (BLEU-2 to BLEU-5, p<1e-6).
• LeakGAN outperforms baselines on COCO image captions (BLEU-2 to BLEU-4, p<1e-6).
• LeakGAN improves BLEU scores on Chinese poems (BLEU-2, p<1e-6).
• Turing tests show LeakGAN's generated text more often indistinguishable from real text than baselines (p<1e-6).
• Visual analysis indicates leaked features guide the generator toward real data feature regions and reveals learned structural cues without supervision.