[Paper Review] AUSN: Approximately Uniform Quantization by Adaptively Superimposing Non-uniform Distribution for Deep Neural Networks

Fangxin Liu, Wenbo Zhao+3 more|arXiv (Cornell University)|Jul 8, 2020

Image and Signal Denoising Methods31 references4 citations

TL;DR

AUSN proposes a novel quantization framework that adaptively superimposes non-uniformly quantized values to approximate uniform quantization, enabling high-accuracy, low-bitwidth inference in DNNs without retraining. It achieves up to 2× reduction in FPGA energy and 4.4× in LUTs/FFs by eliminating decoders and using shift-add operations, while maintaining state-of-the-art accuracy across image, speech, and object detection tasks at 4–5 bits.

ABSTRACT

Quantization is essential to simplify DNN inference in edge applications. Existing uniform and non-uniform quantization methods, however, exhibit an inherent conflict between the representing range and representing resolution, and thereby result in either underutilized bit-width or significant accuracy drop. Moreover, these methods encounter three drawbacks: i) the absence of a quantitative metric for in-depth analysis of the source of the quantization errors; ii) the limited focus on the image classification tasks based on CNNs; iii) the unawareness of the real hardware and energy consumption reduced by lowering the bit-width. In this paper, we first define two quantitative metrics, i.e., the Clipping Error and rounding error, to analyze the quantization error distribution. We observe that the boundary- and rounding- errors vary significantly across layers, models and tasks. Consequently, we propose a novel quantization method to quantize the weight and activation. The key idea is to Approximate the Uniform quantization by Adaptively Superposing multiple Non-uniform quantized values, namely AUSN. AUSN is consist of a decoder-free coding scheme that efficiently exploits the bit-width to its extreme, a superposition quantization algorithm that can adapt the coding scheme to different DNN layers, models and tasks without extra hardware design effort, and a rounding scheme that can eliminate the well-known bit-width overflow and re-quantization issues. Theoretical analysis~(see Appendix A) and accuracy evaluation on various DNN models of different tasks show the effectiveness and generalization of AUSN. The synthesis~(see Appendix B) results on FPGA show $2 imes$ reduction of the energy consumption, and $2 imes$ to $4 imes$ reduction of the hardware resource.

Motivation & Objective

• To address the inherent trade-off between representing range and resolution in low-bit quantization, which causes accuracy drops or underutilized bit-width.
• To develop a unified quantization framework applicable across diverse DNN tasks—image classification, speech recognition, and object detection—beyond CNNs.
• To eliminate decoder overhead and reduce hardware cost and energy consumption in edge accelerators by designing a decoder-free, shift-add-based coding scheme.
• To provide a quantitative error analysis of quantization via Clipping and Rounding errors, enabling adaptive bit allocation per layer.
• To establish a principled link between quantization bit-width and real hardware/energy efficiency for accelerator design.

Proposed method

• AUSN decomposes bit-width into symbol and data parts, with the data part further split into a 'basic' and 'subdivision' component to control representing range and resolution.
• It uses superposition of multiple non-uniformly quantized values to approximate a uniform quantization, enabling flexible and adaptive representation across layers and models.
• A decoder-free coding scheme replaces multiplication with shift and add operations, significantly reducing hardware complexity and energy consumption.
• An adaptive bit-allocation scheme dynamically redistributes bit-width based on layer-wise weight distributions, minimizing Clipping and Rounding errors.
• A novel rounding scheme prevents bit-width overflow and re-quantization overhead by ensuring consistent precision during inference.
• The method is applied to both weights and activations without retraining, supporting 4–5 bit quantization across diverse models and tasks.

Experimental results

Research questions

• RQ1 How can we balance representing range and resolution in low-bit quantization without sacrificing accuracy or efficiency?
• RQ2 What are the dominant sources of quantization error in existing methods, and how can they be quantitatively analyzed and mitigated?
• RQ3 Can a unified quantization framework be designed to generalize across image, speech, and object detection tasks without retraining?
• RQ4 How can we eliminate decoder overhead while maintaining high-precision representation in low-bit DNN inference?
• RQ5 What is the real hardware and energy cost of low-bit quantization, and how can it be minimized through algorithmic design?

Key findings

• AUSN achieves state-of-the-art accuracy at 4-bit and 5-bit quantization across image classification (e.g., 93.15% top-1 on CIFAR-10), speech recognition (93.17% on Speech Commands), and object detection (31.6% mAP on COCO), with less than 0.5% accuracy drop.
• On FPGA, AUSN reduces energy consumption by 1.9× and LUTs by 2.0×, and FFs by 4.4× compared to baseline 8-bit implementations.
• The method eliminates the need for a decoder, enabling hardware-friendly computation using only shift and add operations, which reduces DSP usage and logic resource overhead.
• The adaptive superposition scheme dynamically allocates bit-width based on layer-specific error characteristics, reducing Clipping and Rounding errors by up to 0.47% in top-1 accuracy.
• Theoretical analysis shows that AUSN minimizes information loss and provides a principled trade-off between quantization precision and hardware efficiency.
• AUSN generalizes across models and tasks, including YOLOv3-tiny, CRNNs, and TDNNs, without requiring retraining or task-specific adaptation.