[Paper Review] Compressing Deep Convolutional Networks using Vector Quantization

Yunchao Gong, Liu Liu+2 more|arXiv (Cornell University)|Dec 18, 2014

Advanced Neural Network Applications20 references1,018 citations

TL;DR

This paper proposes using vector quantization—particularly k-means clustering and product quantization—to compress deep convolutional neural networks, focusing on reducing the size of dense connected layers. It achieves up to 24× compression with less than 1% accuracy drop on ImageNet, enabling deployment of large models on mobile and embedded devices.

ABSTRACT

Deep convolutional neural networks (CNN) has become the most promising method for object recognition, repeatedly demonstrating record breaking results for image classification and object detection in recent years. However, a very deep CNN generally involves many layers with millions of parameters, making the storage of the network model to be extremely large. This prohibits the usage of deep CNNs on resource limited hardware, especially cell phones or other embedded devices. In this paper, we tackle this model storage issue by investigating information theoretical vector quantization methods for compressing the parameters of CNNs. In particular, we have found in terms of compressing the most storage demanding dense connected layers, vector quantization methods have a clear gain over existing matrix factorization methods. Simply applying k-means clustering to the weights or conducting product quantization can lead to a very good balance between model size and recognition accuracy. For the 1000-category classification task in the ImageNet challenge, we are able to achieve 16-24 times compression of the network with only 1% loss of classification accuracy using the state-of-the-art CNN.

Motivation & Objective

• Address the challenge of storing large deep convolutional neural network models on resource-constrained devices like smartphones.
• Overcome the limitation of existing matrix factorization methods that focus on inference speed rather than model size reduction.
• Systematically evaluate vector quantization techniques as an alternative to matrix factorization for compressing dense layers.
• Demonstrate that vector quantization can achieve higher compression ratios while maintaining high accuracy compared to prior methods.
• Verify the generalization of compressed models beyond image classification, including image retrieval tasks.

Proposed method

• Apply k-means clustering to quantize the weights of dense connected layers into discrete codebooks, reducing parameter precision.
• Use product quantization (PQ) to partition weight vectors into subvectors and quantize each independently, enabling high compression with low error.
• Explore residual quantization (RQ) as a structured quantization method to further improve reconstruction accuracy.
• Train the original CNN model first, then quantize only the fully connected layers post-training without fine-tuning.
• Store only the quantized codebooks and indices instead of full-precision weights, drastically reducing model size.
• Use cosine similarity on 2048D features extracted from the last layer for image retrieval evaluation on the Holidays dataset.

Experimental results

Research questions

• RQ1 Can vector quantization methods achieve better compression than matrix factorization for reducing the storage of deep CNNs?
• RQ2 How does k-means-based scalar quantization compare to structured quantization methods like product quantization in terms of compression and accuracy?
• RQ3 To what extent can compressed CNNs maintain performance on downstream tasks like image retrieval?
• RQ4 What is the trade-off between compression ratio and accuracy loss when using different quantization strategies?
• RQ5 Can low-bit quantization (e.g., 1-bit or 2-bit) still preserve sufficient representational capacity for real-world applications?

Key findings

• K-means quantization with 32 centers (5.3 bits per weight) achieved 16× compression with only 0.5% top-5 accuracy drop on ImageNet.
• Product quantization (PQ) achieved up to 24× compression with less than 1% accuracy loss on ImageNet, outperforming matrix factorization methods.
• Even 1-bit k-means quantization (2 centers) achieved 32× compression and mAP of 67.61 on the Holidays dataset, outperforming the original features.
• The compressed models generalized well to image retrieval, with PQ achieving 66.37 mAP at 13.7× compression, close to the original 66.43 mAP.
• The results confirm that useful parameters in CNNs are highly redundant—compression ratios of 20× align with findings that only ~5% of parameters are essential.
• Residual quantization (RQ) performed poorly and was excluded from further analysis, indicating it is less effective for this task.