[Paper Review] Designing Adaptive Neural Networks for Energy-Constrained Image Classification

Dimitrios Stamoulis, Ting-Wu Chin+5 more|arXiv (Cornell University)|Aug 5, 2018

Advanced Neural Network Applications30 references4 citations

TL;DR

This paper proposes a globally optimized adaptive CNN framework that jointly tunes network architectures and selection policies for energy-constrained mobile image classification. By formulating the design as a hyper-parameter optimization problem and enhancing Bayesian optimization with hardware-aware fine-tuning, the method reduces energy consumption by up to 6× compared to prior black-box CNN approaches on a mobile SoC.

ABSTRACT

As convolutional neural networks (CNNs) enable state-of-the-art computer vision applications, their high energy consumption has emerged as a key impediment to their deployment on embedded and mobile devices. Towards efficient image classification under hardware constraints, prior work has proposed adaptive CNNs, i.e., systems of networks with different accuracy and computation characteristics, where a selection scheme adaptively selects the network to be evaluated for each input image. While previous efforts have investigated different network selection schemes, we find that they do not necessarily result in energy savings when deployed on mobile systems. The key limitation of existing methods is that they learn only how data should be processed among the CNNs and not the network architectures, with each network being treated as a blackbox. To address this limitation, we pursue a more powerful design paradigm where the architecture settings of the CNNs are treated as hyper-parameters to be globally optimized. We cast the design of adaptive CNNs as a hyper-parameter optimization problem with respect to energy, accuracy, and communication constraints imposed by the mobile device. To efficiently solve this problem, we adapt Bayesian optimization to the properties of the design space, reaching near-optimal configurations in few tens of function evaluations. Our method reduces the energy consumed for image classification on a mobile device by up to 6x, compared to the best previously published work that uses CNNs as blackboxes. Finally, we evaluate two image classification practices, i.e., classifying all images locally versus over the cloud under energy and communication constraints.

Motivation & Objective

• Address the limitation of prior adaptive CNNs that treat networks as blackboxes, optimizing only the selection policy while ignoring architectural tuning.
• Overcome the inefficacy of existing methods on real mobile platforms by jointly optimizing network architectures and selection schemes under hardware constraints.
• Enable systematic, hardware-measured optimization of adaptive CNNs for mobile deployment, focusing on energy, accuracy, and communication trade-offs.
• Demonstrate that architectural hyper-parameters (e.g., number of filters, hidden units) significantly impact energy efficiency and should be globally optimized.
• Explore the trade-off between local and remote inference, showing that hybrid on-device/cloud execution can yield further energy savings.

Proposed method

• Formulate adaptive CNN design as a hyper-parameter optimization problem, treating network architecture parameters (e.g., number of feature maps, hidden units) as tunable hyper-parameters.
• Apply Bayesian optimization (BO) to efficiently explore the high-dimensional design space, with a novel κ-based fine-tuning step to accelerate convergence in over-constrained regions.
• Use real hardware measurements from a commercial Nvidia Tegra TX1 mobile SoC to guide optimization, ensuring results reflect actual energy consumption.
• Integrate communication cost modeling to evaluate hybrid local-cloud inference strategies, enabling trade-off analysis between computation and data transfer energy.
• Construct Pareto fronts to compare designs across energy, accuracy, and communication constraints, identifying near-optimal configurations.
• Validate the method via exhaustive grid search and compare against state-of-the-art static baselines (e.g., Bolukbasi et al., Park et al.) on CIFAR-10.

Experimental results

Research questions

• RQ1 Can jointly optimizing network architectures and selection policies lead to significant energy savings in adaptive CNNs on mobile platforms?
• RQ2 How does hardware-aware hyper-parameter optimization compare to black-box approaches that fix network architectures?
• RQ3 To what extent can hybrid local-cloud inference reduce energy consumption compared to fully local execution under the same accuracy constraints?
• RQ4 Does incorporating a κ-based fine-tuning step in Bayesian optimization improve convergence speed and solution quality in energy-constrained design spaces?
• RQ5 How does the number of networks in an adaptive system affect energy efficiency and accuracy trade-offs?

Key findings

• The proposed BO⁺ method reduces energy consumption by up to 6× compared to the best previously published black-box CNN approach under accuracy constraints.
• The method achieves a 31.13% improvement in accuracy under energy constraints compared to prior work, demonstrating superior optimization of the accuracy-energy trade-off.
• The κ-based fine-tuning step in Bayesian optimization reduces energy consumption by 13.18% under accuracy constraints and by 21.22% under energy constraints compared to standard BO.
• Hybrid execution—where only the smallest CNN runs locally and larger ones are offloaded—reduces energy use by 2.96× compared to full local execution at the same error level.
• The three-network adaptive design using BO⁺ achieves 113.59 mJ of minimum energy under a 3% accuracy degradation constraint, only 4.74% above the optimal grid search result.
• The Pareto front analysis confirms that the method generalizes well, with no overfitting observed on the validation set, and identifies near-optimal configurations across multiple constraints.