[Paper Review] Performance Guaranteed Network Acceleration via High-Order Residual Quantization

Zefan Li, Bingbing Ni+3 more|arXiv (Cornell University)|Aug 29, 2017

Sparse and Compressive Sensing Techniques20 references19 citations

TL;DR

This paper proposes High-Order Residual Quantization (HORQ), a novel binary approximation method that recursively applies thresholding to reduce quantization error in deep neural networks. By generating multiple binary feature maps at decreasing magnitude scales and using high-order binary filtering, HORQ achieves up to 30× speedup with minimal accuracy drop, outperforming XNOR-Net by ~3% on CIFAR-10 and 0.71% on MNIST while maintaining 32× model compression.

ABSTRACT

Input binarization has shown to be an effective way for network acceleration. However, previous binarization scheme could be regarded as simple pixel-wise thresholding operations (i.e., order-one approximation) and suffers a big accuracy loss. In this paper, we propose a highorder binarization scheme, which achieves more accurate approximation while still possesses the advantage of binary operation. In particular, the proposed scheme recursively performs residual quantization and yields a series of binary input images with decreasing magnitude scales. Accordingly, we propose high-order binary filtering and gradient propagation operations for both forward and backward computations. Theoretical analysis shows approximation error guarantee property of proposed method. Extensive experimental results demonstrate that the proposed scheme yields great recognition accuracy while being accelerated.

Motivation & Objective

• To address the significant accuracy degradation caused by first-order binarization in deep neural networks.
• To develop a binary quantization scheme that maintains high approximation accuracy while enabling efficient binary computation.
• To provide theoretical error guarantee for the proposed quantization framework.
• To enable practical deployment of deep networks on CPU with minimal performance loss.

Proposed method

• Proposes High-Order Residual Quantization (HORQ), which recursively applies thresholding to residual errors to generate multiple binary feature maps at decreasing magnitude scales.
• Introduces high-order binary filtering and gradient propagation operations that support forward and backward computation using only binary operations.
• Employs a recursive residual decomposition: at order K, the input is approximated as the sum of K binary maps and a residual, with each stage refining the approximation.
• Derives theoretical bounds on approximation error, showing that the method guarantees error reduction with increasing quantization order.
• Uses binary weights and binary activation maps to achieve 32× model compression and enables computation using only 64-bit binary operations per cycle on modern CPUs.
• Applies the method to standard convolutional layers, where the final output is the sum of outputs from each residual stage.

Experimental results

Research questions

• RQ1 Can recursive residual quantization reduce approximation error in binary neural networks beyond first-order thresholding?
• RQ2 Can high-order binary operations maintain high accuracy while enabling significant speedup in inference?
• RQ3 What is the theoretical error bound of the proposed high-order residual quantization scheme?
• RQ4 How does the speedup ratio scale with filter size, number of channels, and quantization order?
• RQ5 Can HORQ outperform existing binarization methods like XNOR-Net in both accuracy and efficiency?

Key findings

• HORQ-Net achieves a 30× speedup on average, with speedup ratios of 31.98× under optimal conditions (64×256 channels, 3×3 filters).
• The method provides a theoretical error guarantee, showing that approximation error decreases with increasing quantization order.
• On CIFAR-10, HORQ-Net outperforms XNOR-Net by approximately 3% in top-1 accuracy.
• On MNIST, HORQ-Net achieves 0.71% higher accuracy than XNOR-Net.
• The model achieves ~32× compression ratio using binary weights, significantly reducing storage requirements.
• Speedup remains above 20× for order-two and order-three quantization, making it practical for real-world deployment.