[Paper Review] Dual Generator Generative Adversarial Networks for Multi-Domain Image-to-Image Translation

Hao Tang, Dan Xu+3 more|arXiv (Cornell University)|Jan 14, 2019

Advanced Image Processing Techniques49 references4 citations

TL;DR

This paper proposes G2GAN, a dual-generator GAN framework for scalable, multi-domain unpaired image-to-image translation using a single model. Unlike prior methods requiring multiple models or sharing a single generator for translation and reconstruction, G2GAN employs task-specific generators (translation and reconstruction) with flexible parameter sharing, achieving superior image quality and stability through joint optimization with color cycle-consistency, SSIM, and identity losses, outperforming state-of-the-art models on six datasets including faces, landscapes, and buildings.

ABSTRACT

State-of-the-art methods for image-to-image translation with Generative Adversarial Networks (GANs) can learn a mapping from one domain to another domain using unpaired image data. However, these methods require the training of one specific model for every pair of image domains, which limits the scalability in dealing with more than two image domains. In addition, the training stage of these methods has the common problem of model collapse that degrades the quality of the generated images. To tackle these issues, we propose a Dual Generator Generative Adversarial Network (G$^2$GAN), which is a robust and scalable approach allowing to perform unpaired image-to-image translation for multiple domains using only dual generators within a single model. Moreover, we explore different optimization losses for better training of G$^2$GAN, and thus make unpaired image-to-image translation with higher consistency and better stability. Extensive experiments on six publicly available datasets with different scenarios, i.e., architectural buildings, seasons, landscape and human faces, demonstrate that the proposed G$^2$GAN achieves superior model capacity and better generation performance comparing with existing image-to-image translation GAN models.

Motivation & Objective

• To address the scalability and model collapse issues in existing unpaired image-to-image translation methods that require training multiple models for each domain pair.
• To improve training stability and generation quality by decoupling translation and reconstruction tasks using two specialized generators instead of sharing one generator.
• To enable efficient multi-domain translation with a single model, reducing the number of required models from Θ(m²) to just one, even for large numbers of domains.
• To explore and integrate multiple optimization losses—color cycle-consistency, multi-scale SSIM, and identity loss—for enhanced training stability and image fidelity.

Proposed method

• G2GAN employs two distinct generators: a translation generator Gᵗ that maps images from source domain X to target domain Y using a target domain label zᵧ, and a reconstruction generator Gʳ that maps the generated image back to the original domain X using the original label zₓ.
• The two generators are trained jointly with a shared discriminator, allowing for adversarial training while maintaining task-specific network architectures and parameter sharing levels.
• A color cycle-consistency loss is introduced to preserve color distribution across the translation and reconstruction paths, improving visual fidelity.
• Multi-scale SSIM and identity loss are incorporated to enhance structural consistency and preserve identity features, especially in face translation tasks.
• The model supports flexible parameter sharing strategies—fully shared, partially shared, or no sharing—enabling trade-offs between model capacity and performance.
• The framework is trained end-to-end using a joint optimization objective combining adversarial loss, cycle-consistency loss, and perceptual losses, stabilizing training and mitigating mode collapse.

Experimental results

Research questions

• RQ1 Can a single-model framework with dual generators outperform existing multi-model or shared-generator approaches in multi-domain image-to-image translation?
• RQ2 Does decoupling translation and reconstruction into separate generators improve training stability and generation quality compared to shared-generator models like StarGAN?
• RQ3 How do different optimization losses—color cycle-consistency, SSIM, and identity loss—affect the performance and stability of the G2GAN framework?
• RQ4 What is the impact of varying levels of parameter sharing between the two generators on model capacity and generation quality?
• RQ5 Can G2GAN achieve state-of-the-art performance across diverse domains such as faces, landscapes, and architectural buildings with a single, unified model?

Key findings

• G2GAN achieves the best FID scores on the Facades, AR, and Bu3dfe datasets, with 23.6% and 55.625% Turkers' accuracy for label-to-photo and photo-to-label translation, respectively, outperforming StarGAN and other baselines.
• The ablation study shows that removing the identity loss (All - I) reduces performance to 2.6% and 4.2% Turkers' accuracy, indicating its importance for identity preservation.
• The variant with double discriminators (All - D) achieves 9.0% and 5.3% Turkers' accuracy, showing improved stability over the full model, though performance drops when combined with other losses.
• G2GAN uses significantly fewer parameters (53.2M–61.6M) than CycleGAN (52.6M×21), DualGAN (178.7M×21), and StarGAN (53.2M×1), while achieving better or comparable performance.
• The full G2GAN model achieves 10.3% Turkers' accuracy on label-to-photo translation and 22.8% on AR dataset, demonstrating strong performance across diverse domains.
• The model with no parameter sharing (G2GAN no-sharing) achieves 61.6M parameters and 2.1% Turkers' accuracy on AR, indicating that parameter sharing is crucial for performance, though the model remains competitive.