[Paper Review] MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Andrew Howard, Menglong Zhu+6 more|arXiv (Cornell University)|Apr 17, 2017

Advanced Neural Network Applications32 references9,892 citations

TL;DR

MobileNets introduce depthwise separable convolutions to build lightweight, low-latency CNNs for mobile and embedded vision, with two simple hyper-parameters (width multiplier and resolution multiplier) to trade off accuracy, size, and speed.

ABSTRACT

We present a class of efficient models called MobileNets for mobile and embedded vision applications. MobileNets are based on a streamlined architecture that uses depth-wise separable convolutions to build light weight deep neural networks. We introduce two simple global hyper-parameters that efficiently trade off between latency and accuracy. These hyper-parameters allow the model builder to choose the right sized model for their application based on the constraints of the problem. We present extensive experiments on resource and accuracy tradeoffs and show strong performance compared to other popular models on ImageNet classification. We then demonstrate the effectiveness of MobileNets across a wide range of applications and use cases including object detection, finegrain classification, face attributes and large scale geo-localization.

Motivation & Objective

• Motivate the need for small, fast vision models suitable for mobile/embedded devices.
• Propose a lightweight architecture based on depthwise separable convolutions to reduce computation and parameters.
• Introduce two global hyper-parameters (width multiplier and resolution multiplier) to trade off latency, accuracy, and model size.
• Empirically evaluate MobileNets on ImageNet and diverse applications to demonstrate resource-accuracy tradeoffs.
• Provide guidance for practitioners to choose model configurations under real-world constraints.

Proposed method

• Adopt depthwise separable convolutions to factorize standard convolutions into a depthwise spatial filter followed by a 1x1 pointwise convolution.
• Construct a MobileNet architecture with 28 layers (including depthwise and pointwise layers) and use batch normalization and ReLU after each layer.
• Introduce width multiplier alpha to scale the number of channels per layer, reducing computation roughly quadratically with alpha.
• Introduce resolution multiplier rho to scale input and all internal representations, reducing computation by rho^2.
• Train with TensorFlow using RMSProp, minimal regularization, and limited data augmentation to suit small models.

Experimental results

Research questions

• RQ1 How does depthwise separable convolution affect accuracy and computational cost compared to standard convolutions?
• RQ2 What are the resource-accuracy tradeoffs when varying width multiplier and resolution multiplier for MobileNets?
• RQ3 Can MobileNets achieve competitive accuracy with significantly reduced parameters and FLOPs on ImageNet and downstream tasks?
• RQ4 How well do MobileNets perform on diverse applications such as object detection, fine-grained recognition, geo-localization, and face attributes?

Key findings

• Depthwise separable convolutions reduce computation by about 8-9x with only about 1% loss in ImageNet accuracy compared to full convolutions.
• Thinner (width-mul) MobileNets can outperform shallower variants at similar compute and parameter counts.
• Accuracy declines smoothly as width multiplier alpha decreases and as input resolution rho decreases, enabling tunable tradeoffs.
• MobileNets achieve competitive accuracy with far fewer parameters and FLOPs than VGG16 and GoogleNet on ImageNet (Table 8).
• Smaller MobileNets enable competitive performance on fine-grained recognition, geolocation, face attributes, object detection, and face embeddings with substantially reduced model size and computation (Tables 9-14).
• Distilled MobileNet variants can retain performance for face attribute classification while using a fraction of multi-adds.