[Paper Review] Fixed Point Quantization of Deep Convolutional Networks

Darryl Dexu Lin, Sachin S. Talathi+1 more|arXiv (Cornell University)|Nov 19, 2015

Advanced Neural Network Applications26 references606 citations

TL;DR

This paper proposes an optimization-based method for fixed-point quantization of deep convolutional networks (DCNs), using signal-to-quantization-noise ratio (SQNR) to determine optimal bit-width allocation across layers. It achieves over 20% model size reduction on CIFAR-10 with no accuracy loss and reports a new state-of-the-art 6.78% error rate after fine-tuning, demonstrating the effectiveness of SQNR-driven bit-width allocation for efficient embedded deployment.

ABSTRACT

In recent years increasingly complex architectures for deep convolution networks (DCNs) have been proposed to boost the performance on image recognition tasks. However, the gains in performance have come at a cost of substantial increase in computation and model storage resources. Fixed point implementation of DCNs has the potential to alleviate some of these complexities and facilitate potential deployment on embedded hardware. In this paper, we propose a quantizer design for fixed point implementation of DCNs. We formulate and solve an optimization problem to identify optimal fixed point bit-width allocation across DCN layers. Our experiments show that in comparison to equal bit-width settings, the fixed point DCNs with optimized bit width allocation offer >20% reduction in the model size without any loss in accuracy on CIFAR-10 benchmark. We also demonstrate that fine-tuning can further enhance the accuracy of fixed point DCNs beyond that of the original floating point model. In doing so, we report a new state-of-the-art fixed point performance of 6.78% error-rate on CIFAR-10 benchmark.

Motivation & Objective

• To address the growing computational and storage demands of deep convolutional networks (DCNs) in embedded and real-time applications.
• To develop a principled method for converting pre-trained floating-point DCNs into fixed-point models without retraining, enabling efficient deployment on resource-constrained hardware.
• To optimize bit-width allocation across DCN layers to minimize model size while preserving classification accuracy.
• To validate that SQNR-based bit-width allocation outperforms uniform bit-width schemes and enables fine-tuning to further improve accuracy.

Proposed method

• Formulates an optimization problem to minimize model size subject to a maximum allowable accuracy degradation, using SQNR as the key metric for quantization noise.
• Derives theoretical SQNR expressions based on weight and activation bit-widths and quantization efficiency (κ), enabling prediction of layer-wise quantization noise.
• Applies a cross-layer bit-width optimization strategy that allocates fewer bits to less sensitive layers and more to critical ones, based on SQNR predictions.
• Employs an offline, analytical computation of inter-layer bit-width relationships to avoid exhaustive search, enabling efficient and scalable deployment.
• Validates SQNR predictions via empirical measurement across multiple layers, showing consistent trends despite minor numerical deviations.
• Uses fine-tuning after quantization to further enhance accuracy, leveraging the regularization effect of quantization noise.

Experimental results

Research questions

• RQ1 Can SQNR-based bit-width allocation outperform uniform bit-width quantization in terms of model size reduction without accuracy loss?
• RQ2 How effective is the proposed optimization method in reducing model size while maintaining classification accuracy on standard benchmarks?
• RQ3 Can fine-tuning after fixed-point conversion further improve accuracy beyond the original floating-point model?
• RQ4 To what extent does the proposed method generalize across different network architectures, such as CIFAR-10 and AlexNet-like networks?

Key findings

• The proposed SQNR-based bit-width optimization reduces model size by over 20% on the CIFAR-10 benchmark without any accuracy loss compared to equal bit-width settings.
• The method achieves a new state-of-the-art fixed-point error rate of 6.78% on CIFAR-10, surpassing the original floating-point model's performance after fine-tuning.
• Fine-tuning after quantization improves accuracy, with the (float, 8b) configuration achieving 6.78% error rate—lower than the original floating-point model's 6.98%.
• Theoretical SQNR predictions closely follow measured values in trend, validating the model's reliability despite minor numerical discrepancies.
• The optimization is most effective when model size is dominated by convolutional layers, and less effective when other components (e.g., fully connected layers) dominate.
• The method enables efficient, offline conversion of any pre-trained DCN into a fixed-point model suitable for embedded deployment, without requiring access to training data or frameworks.