[Paper Review] A Convex Duality Framework for GANs

Farzan Farnia, David Tse|arXiv (Cornell University)|Oct 1, 2018

Sparse and Compressive Sensing Techniques35 citations

TL;DR

This paper introduces a convex duality framework that reinterprets GANs as minimizing a divergence between the generator’s distribution and the data distribution, penalized by moment constraints derived from a constrained discriminator class F. It extends this interpretation to f-GAN and Wasserstein GAN, and proposes a hybrid divergence that changes continuously with the generator, suggesting improved training via Lipschitz regularization.

ABSTRACT

Generative adversarial network (GAN) is a minimax game between a generator mimicking the true model and a discriminator distinguishing the samples produced by the generator from the real training samples. Given an unconstrained discriminator able to approximate any function, this game reduces to finding the generative model minimizing a divergence measure, e.g. the Jensen-Shannon (JS) divergence, to the data distribution. However, in practice the discriminator is constrained to be in a smaller class F such as neural nets. Then, a natural question is how the divergence minimization interpretation changes as we constrain F. In this work, we address this question by developing a convex duality framework for analyzing GANs. For a convex set F, this duality framework interprets the original GAN formulation as finding the generative model with minimum JS-divergence to the distributions penalized to match the moments of the data distribution, with the moments specified by the discriminators in F. We show that this interpretation more generally holds for f-GAN and Wasserstein GAN. As a byproduct, we apply the duality framework to a hybrid of f-divergence and Wasserstein distance. Unlike the f-divergence, we prove that the proposed hybrid divergence changes continuously with the generative model, which suggests regularizing the discriminator's Lipschitz constant in f-GAN and vanilla GAN. We numerically evaluate the power of the suggested regularization schemes for improving GAN's training performance.

Motivation & Objective

• To understand how constraining the discriminator class F alters the divergence minimization interpretation in GANs.
• To develop a unified convex duality framework applicable to f-GAN and Wasserstein GAN.
• To propose a hybrid divergence that combines f-divergence and Wasserstein distance, ensuring continuity with respect to the generator.
• To demonstrate that regularizing the discriminator’s Lipschitz constant improves GAN training stability.
• To numerically evaluate the effectiveness of the proposed regularization schemes.

Proposed method

• Formalizing the GAN minimax game as a duality problem over a convex set F of discriminators.
• Deriving a dual representation where the generator minimizes JS divergence subject to moment constraints induced by F.
• Extending the duality framework to f-GAN by expressing the f-divergence in terms of dual variables from F.
• Adapting the framework to Wasserstein GAN by leveraging Kantorovich-Rubinstein duality.
• Proposing a hybrid divergence that blends f-divergence and Wasserstein distance, ensuring continuity in the generator distribution.
• Deriving theoretical conditions under which the hybrid divergence changes smoothly, justifying Lipschitz regularization.

Experimental results

Research questions

• RQ1 How does constraining the discriminator class F affect the divergence minimization interpretation in GANs?
• RQ2 Can a unified convex duality framework be developed for both f-GAN and Wasserstein GAN?
• RQ3 Does a hybrid divergence combining f-divergence and Wasserstein distance exhibit continuous dependence on the generator?
• RQ4 What is the theoretical justification for regularizing the discriminator’s Lipschitz constant in f-GAN and vanilla GAN?
• RQ5 How effective are the proposed regularization schemes in improving GAN training performance?

Key findings

• The duality framework reveals that constrained GANs minimize JS divergence under moment constraints derived from the discriminator class F.
• The framework generalizes to f-GAN and Wasserstein GAN, providing a common theoretical foundation.
• The proposed hybrid divergence changes continuously with the generator, unlike standard f-divergence, which supports regularization.
• Theoretical analysis shows that Lipschitz regularization stabilizes training by ensuring smooth dependence of the divergence on the generator.
• Numerical results confirm that the suggested regularization schemes improve training performance in practice.