[Paper Review] AMC: AutoML for Model Compression and Acceleration on Mobile Devices

Yihui He, Ji Lin+4 more|arXiv (Cornell University)|Feb 10, 2018

Machine Learning and Data Classification92 references366 citations

TL;DR

AMC uses reinforcement learning (DDPG) to automatically learn layer-wise compression policies, achieving better accuracy/latency trade-offs than hand-crafted methods and delivering significant speedups on mobile and GPU hardware.

ABSTRACT

Model compression is a critical technique to efficiently deploy neural network models on mobile devices which have limited computation resources and tight power budgets. Conventional model compression techniques rely on hand-crafted heuristics and rule-based policies that require domain experts to explore the large design space trading off among model size, speed, and accuracy, which is usually sub-optimal and time-consuming. In this paper, we propose AutoML for Model Compression (AMC) which leverage reinforcement learning to provide the model compression policy. This learning-based compression policy outperforms conventional rule-based compression policy by having higher compression ratio, better preserving the accuracy and freeing human labor. Under 4x FLOPs reduction, we achieved 2.7% better accuracy than the handcrafted model compression policy for VGG-16 on ImageNet. We applied this automated, push-the-button compression pipeline to MobileNet and achieved 1.81x speedup of measured inference latency on an Android phone and 1.43x speedup on the Titan XP GPU, with only 0.1% loss of ImageNet Top-1 accuracy.

Motivation & Objective

• Motivate efficient deployment of neural networks on mobile devices under latency and resource constraints.
• Automate the search for per-layer compression policies to maximize accuracy under hardware budgets.
• Demonstrate the generality of the approach across networks (VGG, ResNet, MobileNet) and tasks (classification to detection).
• Offer two reward schemes for resource-constrained and accuracy-guaranteed compression.

Proposed method

• Model compression is framed as a layer-wise continuous action control problem.
• A DDPG agent processes a 11-feature layer embedding to output a precise sparsity ratio a_t in (0,1].
• Compression is performed per layer without fine-tuning to quickly estimate final accuracy.
• Rewards combine accuracy with hardware metrics (FLOPs or params) and two protocols are defined: resource-constrained and accuracy-guaranteed.
• Evaluation uses pre-trained networks, with final fine-tuning after policy search for best performance.

Experimental results

Research questions

• RQ1 Can a reinforcement learning agent discover per-layer compression policies that outperform hand-crafted heuristics?
• RQ2 Do continuous per-layer sparsity actions enable finer, more effective model shrinking than discrete choices?
• RQ3 Can AMC generalize compression policies across architectures and tasks (classification to detection)?
• RQ4 Do resource-constrained and accuracy-guaranteed reward schemes reliably achieve target budgets without sacrificing performance?
• RQ5 What are the real-world speedups and accuracy implications on mobile devices and GPUs?

Key findings

• Under 4× FLOPs reduction, AMC achieves 2.7% higher top-1 accuracy than hand-crafted policy for VGG-16 on ImageNet.
• AMC compresses MobileNet to 1.81× faster Android inference and 1.53× faster Titan XP GPU inference with only 0.1% ImageNet Top-1 loss.
• For ResNet-50, AMC increases expert-tuned 3.4× compression to 5× without accuracy loss on ImageNet.
• AMC achieves 1.95× speedup on Google Pixel 1 and up to 1.53–1.95× gains on mobile/GPU with maintained accuracy, outperforming heuristic methods.
• AMC generalizes to object detection: 4× pruning on Faster R-CNN with VGG-16 yields mAP comparable to or better than hand-crafted pruning under the same compression.
• On CIFAR-10, AMC outperforms hand-crafted policies for Plain-20 and ResNet-56 across FLOPs and parameter budgets.