[Paper Review] Convolutional Neural Network Quantization using Generalized Gamma Distribution

Doyun Kim, Han Young Yim+3 more|arXiv (Cornell University)|Oct 31, 2018

Advanced Neural Network Applications17 references4 citations

TL;DR

This paper proposes a novel quantization method for convolutional neural networks (CNNs) that uses the generalized gamma distribution (GGD) to model feature-map activations and optimize quantization step sizes for maximum signal-to-quantization-noise ratio (SQNR). It separately optimizes weights and biases via fractional length selection and applies a backward-forward tuning (BFT) procedure to further improve accuracy, achieving near-floating-point performance with low-precision inference on models like GoogLeNet, AlexNet, and VGG-16.

ABSTRACT

As edge applications using convolutional neural networks (CNN) models grow, it is becoming necessary to introduce dedicated hardware accelerators in which network parameters and feature-map data are represented with limited precision. In this paper we propose a novel quantization algorithm for energy-efficient deployment of the hardware accelerators. For weights and biases, the optimal bit length of the fractional part is determined so that the quantization error is minimized over their distribution. For feature-map data, meanwhile, their sample distribution is well approximated with the generalized gamma distribution (GGD), and accordingly the optimal quantization step size can be obtained through the asymptotical closed form solution of GGD. The proposed quantization algorithm has a higher signal-to-quantization-noise ratio (SQNR) than other quantization schemes previously proposed for CNNs, and even can be more improved by tuning the quantization parameters, resulting in efficient implementation of the hardware accelerators for CNNs in terms of power consumption and memory bandwidth.

Motivation & Objective

• To address the challenge of deploying low-precision CNNs on edge devices with limited power and memory bandwidth.
• To minimize quantization error in weights, biases, and feature-maps without requiring retraining.
• To develop a hardware-friendly quantization scheme that maximizes SQNR while maintaining high inference accuracy.
• To enable efficient deployment of CNN accelerators through optimized bit-width allocation and quantization parameter tuning.

Proposed method

• Models feature-map distributions across CNN layers using the generalized gamma distribution (GGD) to enable asymptotic closed-form derivation of optimal quantization step sizes.
• Optimizes fractional length for weights and biases by minimizing mean squared quantization error over their empirical distributions.
• Applies a backward-forward tuning (BFT) algorithm to refine quantization parameters post-quantization, improving accuracy without retraining.
• Uses a two-stage process: first, SQNR-maximizing quantization for weights and feature-maps; second, BFT to align quantization with actual network performance.
• Employs a non-uniform quantization strategy tailored to the statistical properties of each layer’s weight and activation distributions.
• Supports mixed-precision quantization, allowing different bit-widths for weights and feature-maps (e.g., w8/fm4), enabling trade-offs between accuracy and resource usage.

Experimental results

Research questions

• RQ1 Can the generalized gamma distribution (GGD) accurately model the statistical distribution of feature-map activations across CNN layers?
• RQ2 Does GGD-based quantization yield higher SQNR than uniform or other distribution-based quantization schemes for CNN feature-maps?
• RQ3 Can backward-forward tuning (BFT) effectively improve inference accuracy after quantization without retraining?
• RQ4 How does the proposed method compare to prior quantization schemes in terms of accuracy and energy efficiency at low bit-widths?

Key findings

• The proposed method achieves 68.6% Top-1 accuracy on GoogLeNet with 8-bit weights and 4-bit feature-maps, only 4.4% degradation from floating-point.
• With 6-bit weights and 6-bit feature-maps, the method achieves 66.7% Top-1 accuracy on GoogLeNet, showing strong performance at low bit-widths.
• For AlexNet, the method achieves 56.2% Top-1 accuracy with 8-bit weights and 4-bit feature-maps, matching floating-point performance within 0.6%.
• On VGG-16, the method achieves 68.3% Top-1 accuracy with 8-bit weights and 4-bit feature-maps, matching the floating-point baseline.
• The BFT tuning step improves accuracy by up to 1.9% over baseline quantization, demonstrating its effectiveness in bridging the SQNR-accuracy gap.
• The method outperforms Ristretto and QNN in accuracy loss at 6-bit and 8-bit precision, with only 0.3% loss at 8-bit weights and 4-bit feature-maps.