[Paper Review] Multi-marginal Wasserstein GAN

Jiezhang Cao, Langyuan Mo+4 more|arXiv (Cornell University)|Nov 3, 2019

Multimodal Machine Learning Applications35 citations

TL;DR

MWGAN introduces a multi-marginal Wasserstein GAN framework to jointly minimize Wasserstein distance across a source and multiple target domains, leveraging a shared discriminative potential and cross-domain constraints for improved multi-domain image translation.

ABSTRACT

Multiple marginal matching problem aims at learning mappings to match a source domain to multiple target domains and it has attracted great attention in many applications, such as multi-domain image translation. However, addressing this problem has two critical challenges: (i) Measuring the multi-marginal distance among different domains is very intractable; (ii) It is very difficult to exploit cross-domain correlations to match the target domain distributions. In this paper, we propose a novel Multi-marginal Wasserstein GAN (MWGAN) to minimize Wasserstein distance among domains. Specifically, with the help of multi-marginal optimal transport theory, we develop a new adversarial objective function with inner- and inter-domain constraints to exploit cross-domain correlations. Moreover, we theoretically analyze the generalization performance of MWGAN, and empirically evaluate it on the balanced and imbalanced translation tasks. Extensive experiments on toy and real-world datasets demonstrate the effectiveness of MWGAN.

Motivation & Objective

• Address the multi-marginal matching problem to map a source domain to multiple target domains.
• Overcome inefficiencies and distribution mismatching in pairwise/domain-wise translation methods.
• Exploit cross-domain correlations via a shared discriminative potential and multi-domain OT theory.
• Provide a dual formulation that makes optimization tractable and enables GAN-based learning.
• Analyze generalization performance for multi-domain translation and validate on toy and real datasets.

Proposed method

• Formulate MWGAN using a dual multi-marginal OT problem with inner- and inter-domain constraints.
• Adopt a shared Kantorovich potential f across domains to enable tractable optimization.
• Define the multi-marginal Wasserstein distance W using a maximization over f with domain-specific weights λ_i.
• Train a discriminator f and multiple generators g_i to optimize the MWGAN objective.
• Incorporate an auxiliary domain classifier φ and a mutual information term to enforce inner-domain constraints.
• Introduce inter-domain gradient penalties to relax strict inter-domain constraint enforcement and capture cross-domain correlations.

Experimental results

Research questions

• RQ1 How can we measure and optimize a multi-marginal Wasserstein distance across a source and multiple target domains?
• RQ2 Can a shared potential function effectively exploit cross-domain correlations to improve multi-domain translation?
• RQ3 What is the generalization behavior of MWGAN in multi-domain translation settings?
• RQ4 How do inner-domain and inter-domain constraints affect translation quality across imbalanced domain pairs?

Key findings

• MWGAN achieves lower FID and competitive or superior attribute classification accuracy compared with CycleGAN, UFDN, and StarGAN on CelebA attribute translation tasks (single and multi-attribute).
• MWGAN demonstrates strong performance in imbalanced edge-to-CelebA translation, yielding the lowest FID and naturalistic results.
• On toy distributions, MWGAN closely matches target distributions and provides meaningful discriminator gradients unlike some baselines.
• MWGAN shows favorable qualitative and quantitative results on painting style transfer, handling highly imbalanced domain sets.
• The paper provides a theoretical generalization bound indicating MWGAN can generalize well with sufficient domain samples.