[论文解读] Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding

Song Han, Huizi Mao+1 more|arXiv (Cornell University)|Oct 1, 2015

Advanced Neural Network Applications参考文献 22被引用 3,526

一句话总结

Introduces a three-stage pipeline—pruning, trained quantization with weight sharing, and Huffman coding—to compress deep networks with no accuracy loss, enabling on-chip storage and energy efficiency.

ABSTRACT

Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources. To address this limitation, we introduce "deep compression", a three stage pipeline: pruning, trained quantization and Huffman coding, that work together to reduce the storage requirement of neural networks by 35x to 49x without affecting their accuracy. Our method first prunes the network by learning only the important connections. Next, we quantize the weights to enforce weight sharing, finally, we apply Huffman coding. After the first two steps we retrain the network to fine tune the remaining connections and the quantized centroids. Pruning, reduces the number of connections by 9x to 13x; Quantization then reduces the number of bits that represent each connection from 32 to 5. On the ImageNet dataset, our method reduced the storage required by AlexNet by 35x, from 240MB to 6.9MB, without loss of accuracy. Our method reduced the size of VGG-16 by 49x from 552MB to 11.3MB, again with no loss of accuracy. This allows fitting the model into on-chip SRAM cache rather than off-chip DRAM memory. Our compression method also facilitates the use of complex neural networks in mobile applications where application size and download bandwidth are constrained. Benchmarked on CPU, GPU and mobile GPU, compressed network has 3x to 4x layerwise speedup and 3x to 7x better energy efficiency.

研究动机与目标

• Reduce storage and memory bandwidth requirements of deep neural networks for mobile and embedded deployment.
• Maintain original accuracy while significantly compressing model parameters.
• Enable on-chip SRAM caching by shrinking model size to fit on-chip memory.
• Demonstrate compression gains across multiple architectures (LeNet, AlexNet, VGG-16) on ImageNet and MNIST.

提出的方法

• Perform network pruning to remove low-importance connections and retrain the remaining weights.
• Apply trained quantization to create weight sharing by clustering weights and storing a small codebook plus indices.
• Retrain to fine-tune the shared weights after quantization.
• Apply Huffman coding to exploit non-uniform weight and index distributions for additional compression.
• Evaluate compression on MNIST and ImageNet benchmarks, reporting storage savings and accuracy.

实验结果

研究问题

• RQ1 Can pruning remove redundant connections without sacrificing accuracy in large CNNs?
• RQ2 How much can weight sharing via trained quantization reduce storage while preserving performance?
• RQ3 Does Huffman coding provide additional compression beyond pruning and quantization, and by how much?
• RQ4 What are the practical storage, speed and energy implications of Deep Compression on real hardware?
• RQ5 How do these techniques interact across architectures (LeNet, AlexNet, VGG-16) and datasets (MNIST, ImageNet)?

主要发现

• Achieves 35× to 49× reduction in model storage without accuracy loss across networks.
• AlexNet compresses from 240MB to 6.9MB (35×); VGG-16 from 552MB to 11.3MB (49×).
• Pruning alone reduces parameters by 9× to 13×; quantization reduces per-connection bits from 32 to as low as 5; Huffman coding adds a further 20%–30% compression.
• Pruning and quantization are complementary and can be combined to reach down to about 3% of original size without accuracy loss.
• Compression enables on-chip SRAM storage, reducing energy and enabling mobile deployment; non-batched inference shows 3×–4× speedup and 3×–7× energy efficiency gains.