[论文解读] On distinguishability criteria for estimating generative models

Ian Goodfellow|arXiv (Cornell University)|Dec 19, 2014

Generative Adversarial Networks and Image Synthesis参考文献 2被引用 87

一句话总结

本文分析了噪声对比估计（NCE）、生成对抗网络（GANs）与最大似然估计（MLE）之间的理论关系。结果表明，NCE的动态生成器变体等价于MLE，但GANs在不放弃可区分性博弈的前提下无法恢复MLE梯度，且现有理论无法保证GAN在非凸设置下的收敛性。

ABSTRACT

Two recently introduced criteria for estimation of generative models are both based on a reduction to binary classification. Noise-contrastive estimation (NCE) is an estimation procedure in which a generative model is trained to be able to distinguish data samples from noise samples. Generative adversarial networks (GANs) are pairs of generator and discriminator networks, with the generator network learning to generate samples by attempting to fool the discriminator network into believing its samples are real data. Both estimation procedures use the same function to drive learning, which naturally raises questions about how they are related to each other, as well as whether this function is related to maximum likelihood estimation (MLE). NCE corresponds to training an internal data model belonging to the {\em discriminator} network but using a fixed generator network. We show that a variant of NCE, with a dynamic generator network, is equivalent to maximum likelihood estimation. Since pairing a learned discriminator with an appropriate dynamically selected generator recovers MLE, one might expect the reverse to hold for pairing a learned generator with a certain discriminator. However, we show that recovering MLE for a learned generator requires departing from the distinguishability game. Specifically: (i) The expected gradient of the NCE discriminator can be made to match the expected gradient of MLE, if one is allowed to use a non-stationary noise distribution for NCE, (ii) No choice of discriminator network can make the expected gradient for the GAN generator match that of MLE, and (iii) The existing theory does not guarantee that GANs will converge in the non-convex case. This suggests that the key next step in GAN research is to determine whether GANs converge, and if not, to modify their training algorithm to force convergence.

研究动机与目标

• 阐明生成建模背景下NCE、GANs与MLE之间的理论关系。
• 研究GANs是否能够恢复最大似然估计的梯度更新。
• 考察在可区分性博弈框架下GANs的收敛性特性。
• 识别尽管理论一致，GANs在实践中仍可能欠拟合的原因。
• 评估可区分性博弈函数是否可用于通过对抗训练实现MLE。

提出的方法

• 比较NCE与GANs中使用的可区分性博弈价值函数，其定义为 $ V(p_c, p_g) = \mathbb{E}_{\mathbf{x} \sim p_d} \log p_c(y=1|\mathbf{x}) + \mathbb{E}_{\mathbf{x} \sim p_g} \log p_c(y=0|\mathbf{x}) $。
• 分析使用固定噪声分布的NCE，并证明NCE的动态生成器版本等价于MLE。
• 推导GANs中生成器的期望梯度，并与MLE梯度进行比较，揭示其根本性不匹配。
• 使用逻辑斯蒂函数建模判别器输出 $ p_c(y=1|\mathbf{x}) = \sigma(a(\mathbf{x})) $，并推导生成器损失函数。
• 证明MLE梯度要求 $ f(\mathbf{x}) = -\frac{p_d(\mathbf{x})}{p_g(\mathbf{x})} $，而GAN损失使用 $ f(\mathbf{x}) = -\zeta(a(\mathbf{x})) $，其中 $ \zeta $ 为软plus函数。
• 指出MLE梯度估计的高方差源于判别器不够自信时，导致训练不稳定，需依赖额外机制。

实验结果

研究问题

• RQ1 是否可通过引入动态生成器的NCE变体恢复最大似然估计？
• RQ2 是否存在一种方法可使GANs产生与最大似然估计等价的梯度？
• RQ3 为何GANs在理论上一致的情况下仍常在实践中无法收敛？
• RQ4 可区分性博弈与最大似然估计之间存在何种关系？
• RQ5 是否可通过修改可区分性博弈以确保在非凸设置下的收敛性？

主要发现

• 采用动态生成器的NCE变体在数学上等价于最大似然估计。
• 无论判别器网络如何选择，GAN生成器的期望梯度均无法与MLE梯度匹配。
• GANs中可区分性博弈的成本函数导致的梯度与MLE不同，具体表现为使用软plus函数而非MLE所需的指数函数。
• 在可区分性博弈框架下，MLE梯度估计的高方差源于判别器仅在高度自信时才提供显著梯度，而这对未训练的生成器而言极为罕见。
• 在非凸博弈中基于梯度的学习存在非收敛性，这可能是GANs出现欠拟合现象的合理解释，尽管当前理论框架中尚无收敛性保证。
• 本文建议未来工作应聚焦于确保GAN训练的收敛性，可能通过修改训练算法以强制实现均衡计算。