[Paper Review] MixConv: Mixed Depthwise Convolutional Kernels

Mingxing Tan, Quoc V. Le|arXiv (Cornell University)|Jul 22, 2019

Advanced Neural Network Applications31 references295 citations

TL;DR

MixConv introduces mixed depthwise convolution that applies multiple kernel sizes within a single depthwise operation, yielding better accuracy and efficiency for MobileNets and enabling MixNets via neural architecture search.

ABSTRACT

Depthwise convolution is becoming increasingly popular in modern efficient ConvNets, but its kernel size is often overlooked. In this paper, we systematically study the impact of different kernel sizes, and observe that combining the benefits of multiple kernel sizes can lead to better accuracy and efficiency. Based on this observation, we propose a new mixed depthwise convolution (MixConv), which naturally mixes up multiple kernel sizes in a single convolution. As a simple drop-in replacement of vanilla depthwise convolution, our MixConv improves the accuracy and efficiency for existing MobileNets on both ImageNet classification and COCO object detection. To demonstrate the effectiveness of MixConv, we integrate it into AutoML search space and develop a new family of models, named as MixNets, which outperform previous mobile models including MobileNetV2 [20] (ImageNet top-1 accuracy +4.2%), ShuffleNetV2 [16] (+3.5%), MnasNet [26] (+1.3%), ProxylessNAS [2] (+2.2%), and FBNet [27] (+2.0%). In particular, our MixNet-L achieves a new state-of-the-art 78.9% ImageNet top-1 accuracy under typical mobile settings (<600M FLOPS). Code is at https://github.com/ tensorflow/tpu/tree/master/models/official/mnasnet/mixnet

Motivation & Objective

• Motivate the impact of kernel size in depthwise convolutions and identify limitations of single-kernel designs.
• Propose MixConv to mix multiple kernel sizes within one depthwise convolution.
• Demonstrate MixConv as a drop-in replacement that improves accuracy and efficiency on ImageNet and COCO.
• Develop MixNets via neural architecture search to achieve state-of-the-art mobile performance.
• Show transfer learning effectiveness of MixNets on standard datasets.

Proposed method

• Define MixConv as partitioning input channels into groups and applying distinct kernel sizes per group.
• Restrict kernel sizes per group to increasing odd sizes (3x3, 5x5, 7x7, 9x9, ...).
• Explore channel partition strategies (equal vs exponential) and compare with dilated alternatives.
• Institute MixConv as a drop-in replacement for vanilla depthwise convolution in MobileNets.
• Use neural architecture search (MnasNet-style) to derive MixNets and evaluate on ImageNet and transfer datasets.

Experimental results

Research questions

• RQ1 Does combining multiple kernel sizes in a single depthwise convolution improve accuracy and efficiency over single-kernel depthwise convolutions?
• RQ2 What are effective design choices for MixConv (group size, kernel sizes per group, channel partitioning) for mobile models?
• RQ3 Can MixConv-based architectures (MixNets) surpass prior mobile nets on ImageNet and transfer datasets under FLOPS constraints?
• RQ4 How does MixConv perform in object detection (COCO) relative to vanilla depthwise conv?
• RQ5 Do very large kernels with MixConv remain beneficial when combined with multi-kernel mixing?

Key findings

• MixConv acts as a simple drop-in replacement for vanilla depthwise convolution and improves accuracy and efficiency on MobileNets for ImageNet and COCO.
• Using multiple kernel sizes mitigates the accuracy drop associated with very large single kernels, enabling larger receptive fields with fewer penalties.
• MixNets, discovered via NAS, outperform MobileNetV2, ShuffleNetV2, MnasNet, ProxylessNAS, and FBNet under similar FLOPS, achieving 78.9% top-1 on ImageNet in MixNet-L.
• MixNet-S and MixNet-M achieve strong transfer learning results on CIFAR-10/100, Pets, and Food-101, with MixNet-M attaining 97.92% accuracy on transfer tasks with much lower FLOPS than ResNet-50.
• Exponential channel partition can reduce parameters/FLOPS while maintaining accuracy, and dilated variants underperform compared to large-kernel MixConv in most cases.