[Paper Review] A High-Performance Adaptive Quantization Approach for Edge CNN Applications

Hsu-Hsun Chin, Ren‐Song Tsay+1 more|arXiv (Cornell University)|Jul 18, 2021

Infrared Target Detection Methodologies54 references4 citations

TL;DR

This paper proposes an adaptive quantization method that dynamically adjusts scaling and shifting factors in 4-bit integer inference to mitigate accuracy loss from biased activations in edge-optimized CNNs. Evaluated on ImageNet, COCO, and PTB benchmarks, the approach achieves state-of-the-art accuracy—surpassing even full-precision models in select cases—while enabling deployment on highly resource-constrained edge devices.

ABSTRACT

Recent convolutional neural network (CNN) development continues to advance the state-of-the-art model accuracy for various applications. However, the enhanced accuracy comes at the cost of substantial memory bandwidth and storage requirements and demanding computational resources. Although in the past the quantization methods have effectively reduced the deployment cost for edge devices, it suffers from significant information loss when processing the biased activations of contemporary CNNs. In this paper, we hence introduce an adaptive high-performance quantization method to resolve the issue of biased activation by dynamically adjusting the scaling and shifting factors based on the task loss. Our proposed method has been extensively evaluated on image classification models (ResNet-18/34/50, MobileNet-V2, EfficientNet-B0) with ImageNet dataset, object detection model (YOLO-V4) with COCO dataset, and language models with PTB dataset. The results show that our 4-bit integer (INT4) quantization models achieve better accuracy than the state-of-the-art 4-bit models, and in some cases, even surpass the golden full-precision models. The final designs have been successfully deployed onto extremely resource-constrained edge devices for many practical applications.

Motivation & Objective

• To address the significant accuracy degradation caused by biased activations in quantized edge CNNs.
• To reduce memory bandwidth and computational costs in edge deployment without sacrificing model performance.
• To develop a dynamic quantization strategy that adapts to task-specific loss during training.
• To enable high-accuracy 4-bit inference on extremely resource-constrained edge devices.
• To outperform existing 4-bit quantization methods and, in some cases, surpass full-precision models.

Proposed method

• The method dynamically adjusts scaling and shifting factors during training based on task loss feedback.
• It introduces a learnable quantization function that adapts per layer using gradient-based optimization.
• The quantization process uses a non-uniform, asymmetric quantization scheme to preserve activation distribution characteristics.
• Scaling and shifting parameters are updated via backpropagation to minimize the task loss during fine-tuning.
• The approach is applied during post-training quantization and fine-tuning, maintaining compatibility with standard CNN architectures.
• The method is evaluated across multiple models: ResNet, MobileNet-V2, EfficientNet-B0, YOLO-V4, and language models on PTB.

Experimental results

Research questions

• RQ1 Can dynamic adjustment of quantization parameters reduce accuracy loss in 4-bit quantized edge CNNs?
• RQ2 How does adaptive scaling and shifting based on task loss compare to fixed quantization schemes?
• RQ3 Can 4-bit quantized models achieve accuracy comparable to or exceeding full-precision models?
• RQ4 To what extent can this method be deployed on highly resource-constrained edge devices?
• RQ5 Does the adaptive quantization method generalize across diverse CNN architectures and tasks?

Key findings

• The proposed 4-bit quantization method achieves higher top-1 accuracy than state-of-the-art 4-bit quantization baselines on ImageNet.
• In several cases, the 4-bit models surpass the accuracy of their full-precision counterparts, particularly on ResNet-18 and MobileNet-V2.
• The method maintains high performance across diverse tasks, including image classification, object detection (YOLO-V4 on COCO), and language modeling (PTB).
• The final quantized models were successfully deployed on extremely resource-constrained edge devices, confirming practical feasibility.
• The adaptive adjustment of scaling and shifting factors significantly reduces information loss from biased activations, especially in deeper and modern CNNs.
• The method demonstrates robustness and generalization across multiple network architectures and datasets.