[Paper Review] QGAN: Quantized Generative Adversarial Networks

Peiqi Wang, Dongsheng Wang+6 more|arXiv (Cornell University)|Jan 24, 2019

Generative Adversarial Networks and Image Synthesis21 references21 citations

TL;DR

This paper proposes QGAN, a novel quantization method for Generative Adversarial Networks (GANs) based on the Expectation-Maximization (EM) algorithm, to overcome the underrepresentation of quantized weight values that plagues existing quantization techniques. QGAN enables effective 1-bit and 2-bit quantization of GANs while preserving high-quality sample generation, outperforming prior methods that fail or produce noisy outputs at low bit-widths.

ABSTRACT

The intensive computation and memory requirements of generative adversarial neural networks (GANs) hinder its real-world deployment on edge devices such as smartphones. Despite the success in model reduction of CNNs, neural network quantization methods have not yet been studied on GANs, which are mainly faced with the issues of both the effectiveness of quantization algorithms and the instability of training GAN models. In this paper, we start with an extensive study on applying existing successful methods to quantize GANs. Our observation reveals that none of them generates samples with reasonable quality because of the underrepresentation of quantized values in model weights, and the generator and discriminator networks show different sensitivities upon quantization methods. Motivated by these observations, we develop a novel quantization method for GANs based on EM algorithms, named as QGAN. We also propose a multi-precision algorithm to help find the optimal number of bits of quantized GAN models in conjunction with corresponding result qualities. Experiments on CIFAR-10 and CelebA show that QGAN can quantize GANs to even 1-bit or 2-bit representations with results of quality comparable to original models.

Motivation & Objective

• To investigate the feasibility and challenges of applying existing neural network quantization methods to GANs, which are known to be unstable under quantization.
• To identify key failure modes in existing quantization techniques when applied to GANs, particularly the underrepresentation of quantized weight values and differential sensitivity between generator and discriminator.
• To develop a novel quantization method, QGAN, that leverages the EM algorithm to better represent original weight distributions in low-bit quantized models.
• To design a multi-precision quantization strategy that optimizes bit-width allocation between generator and discriminator based on their relative sensitivity to quantization.
• To empirically validate that 1-bit and 2-bit quantized GANs can generate samples of quality comparable to full-precision models.

Proposed method

• Proposes QGAN, a quantization method that uses an EM algorithm to cluster full-precision weights into quantized levels, ensuring better distributional alignment and reducing underrepresentation of quantized values.
• Applies a linear scaling function in the EM-based clustering process to improve the utilization of quantized states and stabilize training dynamics in GANs.
• Introduces a multi-precision quantization algorithm that separately quantizes the generator and discriminator with different bit-widths, guided by empirical observations on their sensitivity to quantization.
• Employs a two-stage training strategy: first quantizing the discriminator and then jointly fine-tuning both networks to stabilize convergence.
• Uses Inception Score (IS) and visual inspection to evaluate generated sample quality across different bit-widths and quantization methods.
• Validates the method on CIFAR-10 and CelebA datasets using multiple GAN architectures (DCGAN, WGAN-GP, LSGAN) under varying quantization settings.

Experimental results

Research questions

• RQ1 Why do standard quantization methods fail when applied directly to GANs despite their success in CNNs and RNNs?
• RQ2 How do the generator and discriminator differ in their sensitivity to weight quantization in terms of model convergence and sample quality?
• RQ3 Can an EM-based clustering approach effectively mitigate the underrepresentation of quantized weight values in GANs?
• RQ4 What is the optimal bit-width configuration for generator and discriminator that balances model compression and sample quality?
• RQ5 Can 1-bit or 2-bit quantized GANs generate samples of quality comparable to full-precision models?

Key findings

• Standard quantization methods fail on GANs due to underrepresentation of quantized weight values, leading to poor sample quality or complete failure in generation.
• The discriminator is significantly more sensitive to quantization than the generator, and quantizing both networks jointly leads to more stable training than quantizing only the generator.
• QGAN achieves state-of-the-art performance in low-bit GAN quantization, generating high-quality samples even at 1-bit and 2-bit precision, with Inception Scores of 4.33 (DCGAN on CIFAR-10) and 2.68 (DCGAN on CelebA) at 1-bit D and 2-bit G.
• The multi-precision quantization strategy successfully identifies optimal bit-width configurations, with WGAN-GP achieving 3.17 IS at 4-bit D and 4-bit G, and LSGAN achieving 3.55 IS at 3-bit D and 3-bit G.
• Visual inspection confirms that QGAN-generated samples are perceptually acceptable and comparable to full-precision baselines, especially at 2-bit and above, while other methods produce noise or artifacts at 1-bit.
• The EM-based clustering in QGAN effectively fits the distribution of original weights, as shown by the more uniform and representative distribution of quantized states compared to baseline quantization methods.