[论文解读] Generative Adversarial Networks with Decoder-Encoder Output Noise

Guoqiang Zhong, Wei Gao+2 more|arXiv (Cornell University)|Jul 11, 2018

Generative Adversarial Networks and Image Synthesis参考文献 21被引用 9

一句话总结

本文提出DE-GANs，一种新型生成对抗网络，通过将随机噪声输入预训练的解码器-编码器自编码器，生成更具信息量的潜在向量，从而提升图像生成质量。通过引入变分贝叶斯推断和隐空间损失，DE-GANs加速了收敛并提升了图像质量，在MNIST、CelebA和CIFAR-10数据集上优于DCGANs和VAEs，FID类指标更高，潜在空间结构更平滑。

ABSTRACT

In recent years, research on image generation methods has been developing fast. The auto-encoding variational Bayes method (VAEs) was proposed in 2013, which uses variational inference to learn a latent space from the image database and then generates images using the decoder. The generative adversarial networks (GANs) came out as a promising framework, which uses adversarial training to improve the generative ability of the generator. However, the generated pictures by GANs are generally blurry. The deep convolutional generative adversarial networks (DCGANs) were then proposed to leverage the quality of generated images. Since the input noise vectors are randomly sampled from a Gaussian distribution, the generator has to map from a whole normal distribution to the images. This makes DCGANs unable to reflect the inherent structure of the training data. In this paper, we propose a novel deep model, called generative adversarial networks with decoder-encoder output noise (DE-GANs), which takes advantage of both the adversarial training and the variational Bayesain inference to improve the performance of image generation. DE-GANs use a pre-trained decoder-encoder architecture to map the random Gaussian noise vectors to informative ones and pass them to the generator of the adversarial networks. Since the decoder-encoder architecture is trained by the same images as the generators, the output vectors could carry the intrinsic distribution information of the original images. Moreover, the loss function of DE-GANs is different from GANs and DCGANs. A hidden-space loss function is added to the adversarial loss function to enhance the robustness of the model. Extensive empirical results show that DE-GANs can accelerate the convergence of the adversarial training process and improve the quality of the generated images.

研究动机与目标

• 为解决GAN生成图像模糊和多样性差的问题，通过改进输入噪声的先验分布。
• 将变分推断与对抗训练相结合，以更好地捕捉数据的内在分布。
• 通过结构化噪声先验加速GAN训练的收敛并提升稳定性。
• 在MNIST、CelebA和CIFAR-10等多样化数据集上，提升生成样本的图像质量与语义一致性。

提出的方法

• 使用预训练的解码器-编码器自编码器将标准高斯噪声转换为更能反映数据分布的更具信息量的潜在向量。
• 生成器接收这些优化后的噪声向量，而非原始高斯噪声，从而实现潜在空间的更好初始化。
• 在对抗损失之外引入隐空间损失，以正则化生成器的潜在空间，提升其鲁棒性。
• 自编码器在与GAN相同的训练数据集上进行训练，以确保学习到的先验与数据流形保持一致。
• 模型采用生成器与判别器之间的对抗训练，其中生成器以转换后的噪声为条件。
• 应用主成分分析（PCA）以可视化噪声分布从高斯分布向更结构化、稀疏的后验分布的转变。

实验结果

研究问题

• RQ1 通过预训练自编码器优化随机噪声，能否提升GAN生成图像的质量与多样性？
• RQ2 在GAN噪声空间中引入变分贝叶斯先验，是否能加速训练收敛？
• RQ3 与标准GAN或VAE相比，DE-GANs的潜在空间在平滑性与解耦性方面表现如何？
• RQ4 隐空间损失在多大程度上提升了对抗训练的稳定性与性能？
• RQ5 DE-GANs能否在MNIST、CelebA和CIFAR-10等多样化图像数据集上实现一致的性能提升？

主要发现

• 在CIFAR-10上，使用预训练分类器评估，DE-GANs的测试分类准确率达到78.0%，显著优于DCGANs（75.2%）和VAEs（38.5%）。
• 在MNIST上，99.1%的生成数字被CNN正确分类，表明生成的数字质量高且多样性好。
• 由于解码器-编码器提供的信息性先验，该模型收敛速度优于DCGANs，在仅2个周期内即达到98%的测试准确率。
• PCA可视化显示，后验噪声向量比原始高斯噪声更稀疏、更具结构性，表明先验建模更优。
• DE-GANs的潜在空间表现出更平滑、更连续的结构，支持在CelebA数据集上实现面部图像间的真实插值。
• rdd指标证实，DE-GANs在MNIST上保持了良好的多样性，有效区分手写体与印刷体数字，优于DCGANs。