[论文解读] Adversarial Ranking for Language Generation

Kevin Lin, Dianqi Li+3 more|arXiv (Cornell University)|May 31, 2017

Natural Language Processing Techniques参考文献 24被引用 158

一句话总结

RankGAN 用一个排名器替代二元判别器，在 GAN 框架中对人类撰写的句子进行排序，使其高于机器生成的句子，通过策略梯度训练生成器以产生更高排名的语言输出。

ABSTRACT

Generative adversarial networks (GANs) have great successes on synthesizing data. However, the existing GANs restrict the discriminator to be a binary classifier, and thus limit their learning capacity for tasks that need to synthesize output with rich structures such as natural language descriptions. In this paper, we propose a novel generative adversarial network, RankGAN, for generating high-quality language descriptions. Rather than training the discriminator to learn and assign absolute binary predicate for individual data sample, the proposed RankGAN is able to analyze and rank a collection of human-written and machine-written sentences by giving a reference group. By viewing a set of data samples collectively and evaluating their quality through relative ranking scores, the discriminator is able to make better assessment which in turn helps to learn a better generator. The proposed RankGAN is optimized through the policy gradient technique. Experimental results on multiple public datasets clearly demonstrate the effectiveness of the proposed approach.

研究动机与目标

• Motivation: 通过在 GANs 中超越二元判别器来改进语言生成。
• Aim: 从相对排序信息中学习，以生成更高质量的自然语言。
• Goal: 展示 RankGAN 在多个人公数据集上的有效性，相较于最先进的方法。

提出的方法

• Two-network architecture with a generator G and a ranker R.
• Ranker computes a relative ranking score comparing a candidate sentence to a reference using cosine similarity in embedded space.
• Generator is trained with policy gradient and Monte Carlo rollouts to handle discrete text outputs.
• Ranking score is computed via a softmax-like function over a set of candidates with a reference sentence.
• Training uses a minimax objective that encourages G to produce sentences that rank higher than human-written ones with respect to a reference.
• Ranker training maximizes a ranking objective that contrasts human-written and machine-generated sentences.

实验结果

研究问题

• RQ1 Can a ranking-based discriminator provide richer feedback than a binary classifier for language generation?
• RQ2 Does RankGAN improve generation quality across diverse language tasks and datasets?
• RQ3 How effective is policy gradient with ranking-based rewards for training text generators?
• RQ4 What impact do reference and comparison set sizes have on RankGAN performance?

主要发现

• RankGAN outperforms SeqGAN and other baselines on synthetic data in terms of negative log-likelihood.
• RankGAN achieves higher BLEU-2/BLEU-3/BLEU-4 scores than baselines on Chinese poems, COCO captions, and Shakespeare data.
• Human evaluation scores favor RankGAN-generated text over SeqGAN on Chinese poems and COCO captions.
• RankGAN demonstrates improved language fluency and diversity inferred from both automatic metrics and human judgments.
• The ranking-based objective is more informative than BLEU-based rewards in guiding generator learning.