[Paper Review] Generative Adversarial Network with Multi-Branch Discriminator for Cross-Species Image-to-Image Translation

Ziqiang Zheng, Zhibin Yu+4 more|arXiv (Cornell University)|Jan 24, 2019

Generative Adversarial Networks and Image Synthesis47 references4 citations

TL;DR

This paper proposes GAN-MBD, a multi-branch discriminator architecture that enhances generative adversarial networks for cross-species image-to-image translation by splitting a single discriminator into multiple smaller, parameter-efficient branches. The method improves generation quality and reduces model parameters, enabling high-performance translation across multiple species—including human, cat, and dog—while maintaining efficiency on standard GPUs with minimal memory usage.

ABSTRACT

Current approaches have made great progress on image-to-image translation tasks benefiting from the success of image synthesis methods especially generative adversarial networks (GANs). However, existing methods are limited to handling translation tasks between two species while keeping the content matching on the semantic level. A more challenging task would be the translation among more than two species. To explore this new area, we propose a simple yet effective structure of a multi-branch discriminator for enhancing an arbitrary generative adversarial architecture (GAN), named GAN-MBD. It takes advantage of the boosting strategy to break a common discriminator into several smaller ones with fewer parameters, which can enhance the generation and synthesis abilities of GANs efficiently and effectively. Comprehensive experiments show that the proposed multi-branch discriminator can dramatically improve the performance of popular GANs on cross-species image-to-image translation tasks while reducing the number of parameters for computation. The code and some datasets are attached as supplementary materials for reference.

Motivation & Objective

• To address the challenge of cross-species image-to-image translation, where semantic-level content matching across multiple species is required.
• To reduce the computational burden of existing GANs, which often require high-end GPUs and long training times.
• To enhance the generative and synthesis capabilities of popular GANs without increasing model complexity or parameter count.
• To enable effective translation not only between two species but also among three or more species simultaneously.
• To develop a post-processing refinement technique that improves image quality without adding extra parameters.

Proposed method

• The multi-branch discriminator (MBD) decomposes a single, large discriminator into multiple smaller, parallel branches, each with fewer parameters, inspired by the boosting strategy.
• Each branch processes a subset of features, allowing the model to learn more robust and diverse discriminative representations across species.
• The MBD structure is plug-and-play, compatible with existing GANs such as Pix2pix, CycleGAN, and StarGAN, enhancing their performance without architectural overhaul.
• A recycling and refining post-processing pipeline is introduced to further improve image quality by iteratively refining generated outputs.
• The method reduces total model parameters significantly—e.g., reducing StarGAN’s discriminator from 45.41M to 11.89M with four branches—while improving translation fidelity.
• Experiments use a multi-branch design with 1 to 64 branches, demonstrating scalability and efficiency across different GAN frameworks.

Experimental results

Research questions

• RQ1 Can a multi-branch discriminator structure improve image-to-image translation performance across multiple species while reducing model parameters?
• RQ2 How does the MBD architecture compare to standard GANs and multi-discriminator variants in terms of generation quality and training efficiency?
• RQ3 Can the recycling and refining post-processing method enhance image quality without increasing model parameters?
• RQ4 Does the MBD framework generalize across different GAN architectures, including Pix2pix, CycleGAN, and StarGAN?
• RQ5 Can the proposed method enable effective cross-species translation among three or more species, such as human, cat, and dog?

Key findings

• The MBD structure reduces the total number of parameters in GANs by up to 75%—e.g., StarGAN’s total parameters drop from 58.31M to 24.79M with four branches—while improving performance.
• The proposed GAN-MBD achieves state-of-the-art results in cross-species image-to-image translation, including successful translation among human, cat, and dog using StarGAN-4BR.
• The recycling and refining post-processing step improves image quality in multi-species tasks, as shown in qualitative results on CelebA, Cat2dog, and other datasets.
• Training time is reduced by up to 15% compared to baseline models: for example, StarGAN-4br trains in 315.42ms per 1000 iterations, compared to 373.44ms for standard StarGAN.
• The MBD-based CycleGAN achieves comparable or better image quality than standard CycleGAN with multiple discriminators, but with significantly fewer parameters.
• The method enables high-quality image translation on a single 1080 Ti GPU with limited memory, making it feasible for real-world deployment.