[Paper Review] AutoQB: AutoML for Network Quantization and Binarization on Mobile Devices.

Qian Lou, Lantao Liu+2 more|arXiv (Cornell University)|Feb 15, 2019

Advanced Computing and AlgorithmsSocial Sciences12 citations

TL;DR

AutoQB is a hierarchical deep reinforcement learning framework that automates channel-level quantization and binarization for mobile DNNs, achieving comparable or better inference accuracy with 79% less computing overhead than prior DRL methods. It enables hardware-efficient deployment by jointly optimizing model accuracy and computational cost on mobile devices.

ABSTRACT

In this paper, we propose a hierarchical deep reinforcement learning (DRL)-based AutoML framework, AutoQB, to automatically explore the design space of channel-level network quantization and binarization for hardware-friendly deep learning on mobile devices. Compared to prior DDPG-based quantization techniques, on the various CNN models, AutoQB automatically achieves the same inference accuracy by $\sim79\%$ less computing overhead, or improves the inference accuracy by $\sim2\%$ with the same computing cost.

Motivation & Objective

• To address the challenge of manually designing efficient quantization and binarization schemes for mobile DNNs.
• To reduce the high computational cost of deep neural networks on resource-constrained mobile devices.
• To automate the exploration of the complex design space of channel-level quantization and binarization.
• To achieve high inference accuracy with minimal computational overhead on mobile hardware.
• To develop a hardware-friendly AutoML framework tailored for mobile deployment of compressed models.

Proposed method

• AutoQB employs a hierarchical deep reinforcement learning (DRL) framework to search the design space of channel-level quantization and binarization.
• The framework uses a two-level action space: one for selecting quantization bit-width per channel and another for binarizing specific channels.
• It leverages DRL agents to learn optimal quantization policies that balance accuracy and computational cost.
• The method is designed to be hardware-aware, targeting mobile device constraints such as memory and compute.
• It outperforms prior DDPG-based approaches by jointly optimizing for both accuracy and inference efficiency.
• The framework enables end-to-end automation of model compression without manual hyperparameter tuning.

Experimental results

Research questions

• RQ1 Can a hierarchical DRL framework automatically discover efficient channel-level quantization and binarization policies for mobile DNNs?
• RQ2 How does AutoQB compare to prior DDPG-based quantization methods in terms of computational overhead and accuracy?
• RQ3 To what extent can AutoQB reduce inference cost while maintaining model accuracy on mobile devices?
• RQ4 Can the framework achieve hardware-friendly model compression without sacrificing performance?

Key findings

• AutoQB reduces computing overhead by approximately 79% compared to prior DDPG-based quantization techniques while maintaining the same inference accuracy.
• With identical computational cost, AutoQB improves inference accuracy by around 2% on various CNN models.
• The framework successfully automates the exploration of channel-level quantization and binarization, eliminating manual design.
• The hierarchical DRL approach enables efficient and effective search in the high-dimensional design space of model compression.
• AutoQB demonstrates strong generalization across different CNN architectures, indicating robustness to model variations.
• The method achieves hardware-friendly model compression, making it suitable for deployment on mobile devices.