[Paper Review] Layer-Wise Data-Free CNN Compression

Maxwell Horton, Yanzi Jin+2 more|arXiv (Cornell University)|Nov 18, 2020

Advanced Neural Network Applications4 citations

TL;DR

This paper proposes a layer-wise, data-free method for compressing pre-trained CNNs using only BatchNorm statistics to generate synthetic inputs, enabling efficient quantization and pruning without real data. It achieves state-of-the-art accuracy in 8-bit quantization (+0.34%) and up to 28.50% improvement at 5-bit, while outperforming efficient baselines in pruning and enhancing costly generative methods.

ABSTRACT

We present a computationally efficient method for compressing a trained neural network without using real data. We break the problem of data-free network compression into independent layer-wise compressions. We show how to efficiently generate layer-wise training data using only a pretrained network. We use this data to perform independent layer-wise compressions on the pretrained network. We also show how to precondition the network to improve the accuracy of our layer-wise compression method. We present results for layer-wise compression using quantization and pruning. When quantizing, we compress with higher accuracy than related works while using orders of magnitude less compute. When compressing MobileNetV2 and evaluating on ImageNet, our method outperforms existing methods for quantization at all bit-widths, achieving a $+0.34\%$ improvement in $8$-bit quantization, and a stronger improvement at lower bit-widths (up to a $+28.50\%$ improvement at $5$ bits). When pruning, we outperform baselines of a similar compute envelope, achieving $1.5$ times the sparsity rate at the same accuracy. We also show how to combine our efficient method with high-compute generative methods to improve upon their results.

Motivation & Objective

• To enable efficient, data-free compression of pre-trained CNNs for edge deployment.
• To address the challenge of compressing models when original training data is unavailable due to privacy, legal, or storage constraints.
• To develop a computationally efficient method that maintains high accuracy during quantization and pruning without retraining on real data.
• To improve upon existing data-free methods by enabling layer-wise optimization with synthetic data generation.
• To combine efficiently generated data with high-compute generative methods to further boost performance.

Proposed method

• The method decomposes network compression into independent layer-wise problems, treating each layer as a student network trained via knowledge distillation from the corresponding layer of a pre-trained teacher network.
• Synthetic input data for each layer is generated by inverting BatchNorm statistics from the teacher network, using noise inputs that match the activation statistics of the layer.
• A preconditioning technique, AFCLE (Adaptive Feature-wise Channel-wise Linear Equalization), is applied to stabilize training and improve accuracy during compression.
• The student network is trained using knowledge distillation, where the teacher's output logits guide the student's learning, minimizing the distillation loss.
• For pruning, a sparsity-inducing weight decay is applied, and the method learns layer-wise pruning budgets to guide the sparsity pattern.
• The method can be combined with expensive generative methods (e.g., DI, AKD) by adding the layer-wise loss to their objective, improving their accuracy without requiring real data.

Experimental results

Research questions

• RQ1 Can CNN compression be effectively decoupled into independent layer-wise optimization problems without access to real training data?
• RQ2 How can synthetic data be efficiently generated to approximate the input distribution of a specific layer in a pre-trained network?
• RQ3 What techniques can improve the accuracy of data-free compression, especially under extreme quantization or pruning?
• RQ4 Can a computationally efficient data-free method be combined with high-compute generative methods to improve their performance?
• RQ5 How does the proposed method compare to existing data-free and data-light baselines in terms of accuracy and efficiency?

Key findings

• The method achieves a +0.34% accuracy improvement over prior work in 8-bit quantization of MobileNetV2 on ImageNet, with stronger gains at lower bit-widths.
• At 5-bit quantization, the method achieves a +28.50% relative accuracy improvement over existing data-free methods, demonstrating superior low-bit performance.
• For pruning, the method achieves 1.5 times higher sparsity than comparable efficient baselines at the same accuracy, outperforming global, uniform, and ERK pruning baselines.
• Ablation studies confirm that both input scaling and AFCLE are critical: skipping either reduces accuracy significantly, with input scaling alone causing a 1.5% drop in MobileNetV1 pruning.
• Combining the method with DI or AKD improves their performance, achieving better sparsity-accuracy tradeoffs than either method alone, especially for EfficientNet B0 and MobileNetV2.
• The method maintains high efficiency, using orders of magnitude less compute than high-compute generative methods while achieving better or competitive results.