[Paper Review] A Closer Look at Structured Pruning for Neural Network Compression

Elliot J. Crowley, Jack Turner+2 more|arXiv (Cornell University)|Oct 10, 2018

Advanced Neural Network Applications47 references22 citations

TL;DR

This paper investigates structured pruning for neural network compression and finds that training smaller, reduced-width networks from scratch consistently outperforms pruned-and-fine-tuned models on CIFAR-10 and ImageNet. The authors demonstrate that pruned architectures are easily approximated to create competitive 'copycat' networks, and show that reduced networks achieve significantly faster inference on both CPU and GPU due to better resource efficiency, challenging the necessity of pruning for speed and accuracy.

ABSTRACT

Structured pruning is a popular method for compressing a neural network: given a large trained network, one alternates between removing channel connections and fine-tuning; reducing the overall width of the network. However, the efficacy of structured pruning has largely evaded scrutiny. In this paper, we examine ResNets and DenseNets obtained through structured pruning-and-tuning and make two interesting observations: (i) reduced networks---smaller versions of the original network trained from scratch---consistently outperform pruned networks; (ii) if one takes the architecture of a pruned network and then trains it from scratch it is significantly more competitive. Furthermore, these architectures are easy to approximate: we can prune once and obtain a family of new, scalable network architectures that can simply be trained from scratch. Finally, we compare the inference speed of reduced and pruned networks on hardware, and show that reduced networks are significantly faster. Code is available at https://github.com/BayesWatch/pytorch-prunes.

Motivation & Objective

• To evaluate the effectiveness of structured pruning-and-tuning for neural network compression in comparison to training smaller, reduced networks from scratch.
• To investigate whether the architectures obtained through structured pruning can be repurposed as efficient, trainable architectures (copycat networks) for improved performance.
• To benchmark the inference speed of pruned and reduced networks on standard hardware (CPU and GPU) to assess real-world efficiency.
• To challenge the assumption that structured pruning is necessary for efficient deployment, especially when inference speed and parameter efficiency are prioritized.

Proposed method

• Applying ℓ₁-norm and Fisher pruning to WideResNet and DenseNet architectures on CIFAR-10 and ImageNet datasets.
• Alternating between channel pruning based on saliency metrics and fine-tuning the pruned network to maintain accuracy.
• Creating reduced networks by systematically reducing the width of the original network while training from scratch.
• Deriving 'copycat' architectures from pruned models by preserving the remaining channel structure and retraining from scratch.
• Benchmarking inference speed on CPU (Intel Core i7) and GPU (NVIDIA 1080Ti) for pruned and reduced models under identical conditions.
• Comparing parameter count, MACs, top-1 accuracy, and inference time across all models to evaluate efficiency and performance trade-offs.

Experimental results

Research questions

• RQ1 Does structured pruning-and-tuning produce models that outperform smaller, reduced-width networks trained from scratch?
• RQ2 Can the architectural patterns produced by structured pruning be reused to create competitive models when trained from scratch?
• RQ3 How does the inference speed of pruned networks compare to reduced networks on general-purpose hardware?
• RQ4 What is the impact of pruning on the MACs per parameter ratio, and how does this affect resource efficiency?
• RQ5 Is structured pruning still beneficial when inference speed and parameter efficiency are critical deployment constraints?

Key findings

• For a given parameter budget, reduced networks trained from scratch consistently outperform pruned-and-fine-tuned models on both CIFAR-10 and ImageNet.
• Pruned networks that are retrained from scratch (copycat architectures) surpass both the original pruned-and-fine-tuned models and reduced networks in accuracy, especially in residual networks.
• Reduced networks achieve significantly faster inference times than pruned networks on both CPU and GPU, with MACs per parameter ratios up to 3.5× higher than pruned models.
• The pruned ResNet-34-B, which matches ResNet-18's error rate, has 33% slower inference speed despite having fewer parameters, indicating poor resource efficiency.
• The ResNet-34-A model, with similar parameter count to ResNet-18, has a lower error but is 33% slower in inference, showing that pruning reduces throughput efficiency.
• The study concludes that structured pruning harms inference throughput on general-purpose hardware, making reduced-depth networks a superior alternative for speed and efficiency.