[Paper Review] Self-Supervised GAN Compression

Chong Yu, Jeff Pool|arXiv (Cornell University)|Jul 3, 2020

Generative Adversarial Networks and Image Synthesis42 references4 citations

TL;DR

This paper proposes a self-supervised compression method for Generative Adversarial Networks (GANs) that leverages a pre-trained discriminator to supervise the pruning of a generator, overcoming the failure of standard pruning techniques in maintaining GAN quality. The approach achieves high sparsity (up to 90%) with minimal quality degradation, especially using fine-grained (element-level) pruning, and requires only 1-10% of the original training time.

ABSTRACT

Deep learning's success has led to larger and larger models to handle more and more complex tasks; trained models can contain millions of parameters. These large models are compute- and memory-intensive, which makes it a challenge to deploy them with minimized latency, throughput, and storage requirements. Some model compression methods have been successfully applied to image classification and detection or language models, but there has been very little work compressing generative adversarial networks (GANs) performing complex tasks. In this paper, we show that a standard model compression technique, weight pruning, cannot be applied to GANs using existing methods. We then develop a self-supervised compression technique which uses the trained discriminator to supervise the training of a compressed generator. We show that this framework has a compelling performance to high degrees of sparsity, can be easily applied to new tasks and models, and enables meaningful comparisons between different pruning granularities.

Motivation & Objective

• To address the lack of effective model compression techniques for complex GANs, particularly in generator pruning.
• To identify why standard pruning methods fail to preserve GAN generation quality in complex tasks.
• To develop a self-supervised compression framework that maintains high-fidelity outputs during generator compression.
• To evaluate the impact of different pruning granularities (element-level vs. filter-level) and sparsity levels on GAN performance.
• To enable efficient, end-to-end compression of GAN generators with minimal hyperparameter tuning and retraining.

Proposed method

• The method uses a pre-trained discriminator to provide self-supervision during the compression of a generator, replacing the need for original training data.
• It introduces a novel loss function combining the original and compressed generator outputs with the discriminator’s real/fake predictions.
• Pruning is applied via structured sparsity, with options for element-level (0D), vector-level (1D), kernel-level (2D), and filter-level (3D) compression.
• The compressed generator is fine-tuned using the discriminator’s feedback, enabling high sparsity with minimal performance drop.
• The framework is end-to-end and requires only one optional hyperparameter (set to 0.5 in experiments).
• The approach is applied across multiple GAN architectures and tasks, including image-to-image translation and style transfer.

Experimental results

Research questions

• RQ1 Why do standard pruning techniques fail to maintain the quality of GAN generators in complex tasks?
• RQ2 Can a pre-trained discriminator effectively supervise the compression of a GAN generator without access to original training data?
• RQ3 How do different pruning granularities (element-level vs. filter-level) affect the quality and stability of compressed GAN generators?
• RQ4 What is the optimal trade-off between sparsity and generation quality in compressed GANs?
• RQ5 Can self-supervised compression achieve high compression rates with minimal retraining time and no need for external evaluation metrics?

Key findings

• Fine-grained (element-level) pruning achieved stable performance up to 90% sparsity, with only minor degradation in image quality and FID scores increasing from 32.006 to 41.251.
• Filter-level pruning caused severe artifacts and color shifts even at 25% sparsity, with FID rising to 82.349, indicating poor suitability for GANs.
• The self-supervised compression process required only 1-10% of the original training time, making it highly efficient.
• The method maintained high perceptual quality across diverse tasks, including CycleGAN and StarGAN, with PSNR and SSIM values close to the original generator.
• The approach outperformed prior work in both qualitative and quantitative metrics, especially at high sparsity levels.
• The loss curves accurately reflected final generation quality, enabling reliable monitoring without external metrics.