[Paper Review] Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks

Yang He, Guoliang Kang+3 more|arXiv (Cornell University)|Aug 21, 2018

Advanced Neural Network ApplicationsComputer Science20 references70 citations

TL;DR

Soft Filter Pruning (SFP) prunes filters during training by setting small-norm filters to zero and allowing them to be updated in subsequent epochs, enabling higher model capacity and effective acceleration without heavy reliance on pre-trained models.

ABSTRACT

This paper proposed a Soft Filter Pruning (SFP) method to accelerate the inference procedure of deep Convolutional Neural Networks (CNNs). Specifically, the proposed SFP enables the pruned filters to be updated when training the model after pruning. SFP has two advantages over previous works: (1) Larger model capacity. Updating previously pruned filters provides our approach with larger optimization space than fixing the filters to zero. Therefore, the network trained by our method has a larger model capacity to learn from the training data. (2) Less dependence on the pre-trained model. Large capacity enables SFP to train from scratch and prune the model simultaneously. In contrast, previous filter pruning methods should be conducted on the basis of the pre-trained model to guarantee their performance. Empirically, SFP from scratch outperforms the previous filter pruning methods. Moreover, our approach has been demonstrated effective for many advanced CNN architectures. Notably, on ILSCRC-2012, SFP reduces more than 42% FLOPs on ResNet-101 with even 0.2% top-5 accuracy improvement, which has advanced the state-of-the-art. Code is publicly available on GitHub: https://github.com/he-y/soft-filter-pruning

Motivation & Objective

• Motivation to reduce CNN inference cost while preserving accuracy.
• Develop a pruning method that maintains model capacity by allowing pruned filters to be updated.
• Eliminate the need for layer-by-layer pruning and heavy fine-tuning by integrating pruning into standard training.
• Demonstrate effectiveness across CNN architectures and large-scale datasets (CIFAR-10 and ImageNet).
• Show that SFP can outperform hard pruning methods and can work from scratch or with pre-trained models.

Proposed method

• Prune filters at the end of each training epoch based on their Lp-norm (primarily L2) to select a fraction P of filters per layer.
• Set selected low-importance filters to zero, temporarily removing their contribution during forward passes.
• Continue training so zeroed filters can be reconstructed (updated) via backpropagation, preserving model capacity.
• Prune all weighted layers simultaneously rather than layer-by-layer, using a single pruning rate P across layers.
• Rebuild a compact model after training convergence by discarding zeroed filters and adjusting corresponding input/output channel dimensions.
• Provide a theoretical and empirical analysis of both theoretical and realistic speedups, highlighting that practical gains depend on hardware and libraries.

Experimental results

Research questions

• RQ1 Can soft pruning of filters during training maintain higher model capacity than hard pruning?
• RQ2 Is pruning from scratch viable and competitive with pruning pre-trained models when using SFP?
• RQ3 What is the impact of a uniform pruning rate across layers on accuracy and acceleration?
• RQ4 How does SFP perform on different CNN architectures and datasets (CIFAR-10 and ILSVRC-2012) compared to state-of-the-art pruning methods?

Key findings

• SFP achieves meaningful acceleration (e.g., up to ~42% FLOPs reduction in ResNet-101 with minimal or even positive accuracy change on ILSVRC-2012 under certain settings.
• Soft pruning preserves or even improves accuracy versus baselines when pruning from scratch, and can match or exceed performance when using pre-trained models.
• Compared to hard filter pruning, SFP maintains higher model capacity during training, leading to better accuracy after pruning.
• L2-norm based filter selection is slightly more effective than L1-norm for pruning decisions in their experiments.
• Realistic speedups (actual inference time) are demonstrated to be substantial but can differ from theoretical FLOP-based speedups due to IO and BLAS efficiency.