[Paper Review] Accelerator-aware Neural Network Design using AutoML

Suyog Gupta, Berkin Akin|arXiv (Cornell University)|Mar 5, 2020

CCD and CMOS Imaging SensorsEngineering11 references49 citations

TL;DR

The paper proposes accelerator-aware neural architecture search (NAS) to design edge-optimized vision models for Edge TPU, producing EfficientNet-EdgeTPU and MobilenetEdgeTPU that improve latency-accuracy trade-offs on Coral and Pixel 4 devices. It combines latency modeling, hardware-aware search space design, and NAS to tailor models to specific accelerators.

ABSTRACT

While neural network hardware accelerators provide a substantial amount of raw compute throughput, the models deployed on them must be co-designed for the underlying hardware architecture to obtain the optimal system performance. We present a class of computer vision models designed using hardware-aware neural architecture search and customized to run on the Edge TPU, Google's neural network hardware accelerator for low-power, edge devices. For the Edge TPU in Coral devices, these models enable real-time image classification performance while achieving accuracy typically seen only with larger, compute-heavy models running in data centers. On Pixel 4's Edge TPU, these models improve the accuracy-latency tradeoff over existing SoTA mobile models.

Motivation & Objective

• Motivate on-device ML for privacy-preserving, responsive inference on resource-constrained hardware.
• Demonstrate that hardware-aware NAS can outperform hand-crafted mobile models by co-designing architectures with target accelerators.
• Develop latency estimation methods and a hardware-aware search framework to optimize accuracy and latency.
• Customize the search space to include blocks that maximize Edge TPU utilization while excluding incompatible operations.

Proposed method

• Extend NAS with an accelerator performance predictor that estimates latency on target hardware.
• Use a cycle-accurate Edge TPU simulator to estimate model latency and an analytical performance model (APM) for fast latency estimates.
• Incorporate latency into the NAS objective alongside accuracy in a multi-objective reward.
• Engineer the search space with hardware-aware blocks such as fused inverted bottleneck convolutions to improve utilization.
• Exclude operations not supported by production Edge TPU software to ensure deployability.
• Scale architectures using EfficientNet-style compound scaling to generate variants (-S, -M, -L).

Experimental results

Research questions

• RQ1 Can accelerator-aware NAS discover models that outperform hand-tuned mobile architectures on Edge TPU hardware?
• RQ2 How do latency estimation methods (cycle-level simulation vs. analytical models) impact NAS efficiency and results?
• RQ3 What architectural blocks maximize Edge TPU utilization and accuracy for Coral vs. Pixel 4 deployments?
• RQ4 How do searched models compare against baseline mobile and efficient networks in terms of latency, accuracy, and deployability?

Key findings

• EfficientNet-EdgeTPU-S/M/L achieve faster runtimes and higher accuracy than ResNet50 and Inception on Edge TPU in Coral devices.
• MobilenetEdgeTPU models achieve 75.6% top-1 accuracy on Pixel 4 Edge TPU with 30% lower latency than MobilenetV3.
• NAS-generated models improve the accuracy-latency Pareto frontier on Edge TPU targets compared to existing mobile models.
• On Pixel 4, MobilenetEdgeTPU runs with different latency characteristics than on Coral, highlighting hardware-specific search space needs.
• The accelerator-aware NAS reduces manual architectural engineering by tailoring models to specific Edge TPU variants and production stacks.