[Paper Review] Generative Adversarial Mapping Networks

Jianbo Guo, Guangxiang Zhu+1 more|arXiv (Cornell University)|Sep 28, 2017

Generative Adversarial Networks and Image Synthesis20 references3 citations

TL;DR

This paper proposes Generative Adversarial Mapping Networks (GAMN), a novel GAN-based generative model that enhances moment matching networks by introducing an adversarial mapper network F to improve distribution matching in a feature space. By jointly training the generator G and mapper F to minimize MMD in the representation space, GAMN achieves superior sample quality on CIFAR-10 and LSUN-Bedrooms compared to GMMN and GMMN+AE, with reduced batch size requirements and improved training stability.

ABSTRACT

Generative Adversarial Networks (GANs) have shown impressive performance in generating photo-realistic images. They fit generative models by minimizing certain distance measure between the real image distribution and the generated data distribution. Several distance measures have been used, such as Jensen-Shannon divergence, $f$-divergence, and Wasserstein distance, and choosing an appropriate distance measure is very important for training the generative network. In this paper, we choose to use the maximum mean discrepancy (MMD) as the distance metric, which has several nice theoretical guarantees. In fact, generative moment matching network (GMMN) (Li, Swersky, and Zemel 2015) is such a generative model which contains only one generator network $G$ trained by directly minimizing MMD between the real and generated distributions. However, it fails to generate meaningful samples on challenging benchmark datasets, such as CIFAR-10 and LSUN. To improve on GMMN, we propose to add an extra network $F$, called mapper. $F$ maps both real data distribution and generated data distribution from the original data space to a feature representation space $\mathcal{R}$, and it is trained to maximize MMD between the two mapped distributions in $\mathcal{R}$, while the generator $G$ tries to minimize the MMD. We call the new model generative adversarial mapping networks (GAMNs). We demonstrate that the adversarial mapper $F$ can help $G$ to better capture the underlying data distribution. We also show that GAMN significantly outperforms GMMN, and is also superior to or comparable with other state-of-the-art GAN based methods on MNIST, CIFAR-10 and LSUN-Bedrooms datasets.

Motivation & Objective

• To address the failure of Generative Moment Matching Networks (GMMN) in generating realistic samples on complex datasets like CIFAR-10 and LSUN-Bedrooms.
• To overcome the high batch size requirement and poor sample quality of GMMN and GMMN+AE by introducing a dynamic, adversarially trained mapper network.
• To integrate the theoretical advantages of Maximum Mean Discrepancy (MMD) with the adversarial training paradigm of GANs for more stable and effective distribution learning.
• To develop a simpler yet more powerful alternative to existing MMD-based models that require fixed auto-encoders or complex regularization.
• To demonstrate that an end-to-end, jointly trained mapper network can significantly improve generator performance in generating high-fidelity images.

Proposed method

• Introduce a new generative model, GAMN, which consists of a generator G and a mapper F, both trained jointly in an adversarial manner.
• Map both real data and generated samples from the original data space to a lower-dimensional feature representation space R using the mapper F.
• Train the generator G to minimize the Maximum Mean Discrepancy (MMD) between the real and generated data distributions in the feature space R.
• Train the mapper F to maximize the MMD between the real and generated distributions in R, making it a dynamic adversary to the generator.
• Use a kernel-based MMD estimator to compute the distance between the mapped distributions, enabling efficient and unbiased optimization.
• Train the generator and mapper end-to-end using standard backpropagation, avoiding the need for pre-training or fixed encoders as in GMMN+AE.

Experimental results

Research questions

• RQ1 Can an adversarially trained mapper network improve the distribution matching capability of a generator in a moment matching framework?
• RQ2 Does the joint training of a dynamic mapper with the generator lead to better sample quality than fixed auto-encoders used in GMMN+AE?
• RQ3 Can GAMN achieve state-of-the-art performance on challenging benchmarks like CIFAR-10 and LSUN-Bedrooms while reducing the required batch size?
• RQ4 How does GAMN compare to other MMD-based and GAN-based models in terms of inception score and sample fidelity?
• RQ5 Is the proposed framework more stable and scalable than existing MMD-based generative models, especially in high-dimensional data settings?

Key findings

• GAMN significantly outperforms GMMN and GMMN+AE in generating realistic images on CIFAR-10 and LSUN-Bedrooms, producing samples that are qualitatively superior.
• On the CIFAR-10 dataset, GAMN achieves an inception score of 6.44, which is slightly higher than the 6.24 reported for a recently proposed adversarial kernel learning method.
• The MMD value in the feature space decreases consistently during training, correlating with increasing inception scores and improved sample quality.
• GAMN reduces the required batch size for training compared to GMMN and GMMN+AE, which needed batch sizes as large as 1000.
• On toy datasets such as a mixture of 8 Gaussians and Swiss Roll, GAMN learns the underlying data distribution more accurately than WGAN and improved WGAN.
• The adversarial mapper F enhances the generator’s ability to capture complex data structures, leading to more diverse and realistic samples without requiring additional regularization or fixed encoders.