[论文解读] MAGAN: Margin Adaptation for Generative Adversarial Networks

Ruohan Wang, Antoine Cully+2 more|arXiv (Cornell University)|Apr 12, 2017

Advanced Image Processing Techniques参考文献 16被引用 53

一句话总结

MAGANs 自动根据真实数据的期望能量自适应调整 hinge loss 的边界 m，以稳定 GAN 训练并提高样本质量，在某些假设下具有收敛性保障。

ABSTRACT

We propose the Margin Adaptation for Generative Adversarial Networks (MAGANs) algorithm, a novel training procedure for GANs to improve stability and performance by using an adaptive hinge loss function. We estimate the appropriate hinge loss margin with the expected energy of the target distribution, and derive principled criteria for when to update the margin. We prove that our method converges to its global optimum under certain assumptions. Evaluated on the task of unsupervised image generation, the proposed training procedure is simple yet robust on a diverse set of data, and achieves qualitative and quantitative improvements compared to the state-of-the-art.

研究动机与目标

• Motivate and address instability and mode collapse in GAN training using auto-encoder based discriminators.
• Introduce a margin adaptation mechanism that tunes the hinge loss margin m according to training statistics.
• Provide theoretical convergence analysis showing MAGANs converges to the data distribution under certain conditions.
• Demonstrate empirical improvements in image generation quality and stability across datasets (MNIST, CIFAR-10, CelebA).

提出的方法

• Define D as a deep auto-encoder and use a hinge loss with margin m in the discriminator objective.
• Adapt the margin m_t based on the expected energy of real data to maintain equilibrium between D and G.
• Train with a two-stage process including an initial discriminator pre-training.
• Update m_t when specific statistics indicate stall and divergence tendencies between real and synthetic energies.
• Prove convergence of MAGANs to p_data = p_G under given assumptions and show m_t converges to 0 if distributions diverge.
• Provide a theoretical comparison to EBGANs and BEGANs, highlighting stability and convergence guarantees.

实验结果

研究问题

• RQ1 Can an adaptive margin improve training stability and sample quality in auto-encoder based GANs?
• RQ2 Under what conditions does MAGANs converge to the true data distribution?
• RQ3 How does margin adaptation affect the dynamics between the discriminator and generator compared to fixed-margin approaches?
• RQ4 Are MAGANs robust across diverse datasets (MNIST, CIFAR-10, CelebA) without adding extra hyper-parameters?

主要发现

方法	分数 ± 标准差
Real data	11.24 ± 0.12
DFM	7.72 ± 0.13
EGANs	7.07 ± 0.10
BEGANs	5.62
ALI	5.34 ± 0.05
Improved GANs	4.36 ± 0.04
MIX + WGANs	4.04 ± 0.07
Wasserstein GANs	3.82 ± 0.06
MAGANs	6.40 ± 0.03

• MAGANs achieve qualitative and quantitative improvements over state-of-the-art auto-encoder GAN variants on multiple datasets.
• On CIFAR-10, MAGANs achieve a higher inception score (6.40 ± 0.03) than several baselines (e.g., BEGANs 5.62, EGANs 7.07, Improved GANs 4.36).
• On MNIST, MAGANs produce diverse class coverage and higher inception-like score (7.52 ± 0.03) than EBGANs (7.14 ± 0.04).
• Qualitative CelebA results show more detailed and coherent faces with MAGANs compared to BEGANs and EBGANs.
• Margin adaptation stabilizes training by reducing discriminator power temporarily and guiding the generator to produce lower-energy samples.
• MAGANs converge to p_data ≈ p_G under the stated assumptions, with the margin m_t monotonically decreasing to 0 when distributions do not match.