[Paper Review] DualGAN: Unsupervised Dual Learning for Image-to-Image Translation

Zili Yi, Hao Zhang+2 more|arXiv (Cornell University)|Apr 8, 2017

Generative Adversarial Networks and Image Synthesis1 references310 citations

TL;DR

DualGAN trains two generators in a primal-dual loop using unlabeled image sets from two domains, enabling unsupervised image-to-image translation with reconstruction losses and adversarial discriminators.

ABSTRACT

Conditional Generative Adversarial Networks (GANs) for cross-domain image-to-image translation have made much progress recently. Depending on the task complexity, thousands to millions of labeled image pairs are needed to train a conditional GAN. However, human labeling is expensive, even impractical, and large quantities of data may not always be available. Inspired by dual learning from natural language translation, we develop a novel dual-GAN mechanism, which enables image translators to be trained from two sets of unlabeled images from two domains. In our architecture, the primal GAN learns to translate images from domain U to those in domain V, while the dual GAN learns to invert the task. The closed loop made by the primal and dual tasks allows images from either domain to be translated and then reconstructed. Hence a loss function that accounts for the reconstruction error of images can be used to train the translators. Experiments on multiple image translation tasks with unlabeled data show considerable performance gain of DualGAN over a single GAN. For some tasks, DualGAN can even achieve comparable or slightly better results than conditional GAN trained on fully labeled data.

Motivation & Objective

• Motivate unsupervised image-to-image translation when labeled pairs are scarce or unavailable.
• Develop a dual learning framework inspired by NLP dual learning to train bidirectional translators from unlabeled data.
• Leverage adversarial discriminators and reconstruction loss to enforce domain consistency and generate high-quality translations.

Proposed method

• Use two GANs (primal and dual) to learn G_A: U -> V and G_B: V -> U from unlabeled U and V.
• Employ Wasserstein GAN losses for discriminators D_A and D_B to distinguish real vs. translated samples.
• Add L1 reconstruction losses: ||u - G_B(G_A(u,z), z')|| and ||v - G_A(G_B(v,z'), z)|| to enforce cycle-consistency.
• Configure generators as U-Net style architectures with skip connections to preserve structural details.
• Use PatchGAN discriminators operating on 70x70 patches to capture local textures and styles.
• Train with an alternating scheme: n_critic steps for discriminators, followed by generator updates, using RMSProp and weight clipping as in WGAN.

Experimental results

Research questions

• RQ1 Can two unlabeled image sets from domains U and V train reliable bidirectional translators G_A and G_B without paired data?
• RQ2 Does a reconstruction (cycle-consistency) loss combined with adversarial feedback improve translation quality over single-GAN baselines?
• RQ3 How does DualGAN perform relative to GANs and supervised cGANs across diverse translation tasks without labeled data?
• RQ4 What network design choices (e.g., U-Net generators, PatchGAN discriminators) contribute most to translation fidelity in unsupervised settings?
• RQ5 Are there task-specific limitations where unsupervised translation falls short without some labeled guidance?

Key findings

• DualGAN consistently yields less blur and fewer artifacts than a single GAN across multiple unsupervised translation tasks.
• In several tasks, DualGAN matches or surpasses supervised cGAN performance despite using unlabeled data.
• DualGAN can produce visually convincing translations on unlabeled datasets (e.g., oil to Chinese paintings, plastic to metal, etc.).
• AMT-based perceptual and realness evaluations show DualGAN outperforming GAN and, on some tasks, rivaling cGAN performance.
• For label-to-facade and map-to-aerial tasks, DualGAN lags behind cGAN due to lack of pixel-label correspondence information in training.