[Paper Review] Merging and Evolution: Improving Convolutional Neural Networks for Mobile Applications

Zheng Qin, Zhaoning Zhang+3 more|arXiv (Cornell University)|Mar 24, 2018

Advanced Neural Network Applications25 references3 citations

TL;DR

This paper proposes the Merging-and-Evolution (ME) module and MENet, a family of compact convolutional neural networks for mobile applications. By introducing merging to encode inter-group feature information and evolution to extract spatial patterns, MENet significantly reduces performance degradation from group convolutions, achieving state-of-the-art accuracy under tight computational budgets—surpassing ShuffleNet and MobileNet by up to 4.1% on PASCAL VOC 2007 mAP at 140 MFLOPs.

ABSTRACT

Compact neural networks are inclined to exploit "sparsely-connected" convolutions such as depthwise convolution and group convolution for employment in mobile applications. Compared with standard "fully-connected" convolutions, these convolutions are more computationally economical. However, "sparsely-connected" convolutions block the inter-group information exchange, which induces severe performance degradation. To address this issue, we present two novel operations named merging and evolution to leverage the inter-group information. Our key idea is encoding the inter-group information with a narrow feature map, then combining the generated features with the original network for better representation. Taking advantage of the proposed operations, we then introduce the Merging-and-Evolution (ME) module, an architectural unit specifically designed for compact networks. Finally, we propose a family of compact neural networks called MENet based on ME modules. Extensive experiments on ILSVRC 2012 dataset and PASCAL VOC 2007 dataset demonstrate that MENet consistently outperforms other state-of-the-art compact networks under different computational budgets. For instance, under the computational budget of 140 MFLOPs, MENet surpasses ShuffleNet by 1% and MobileNet by 1.95% on ILSVRC 2012 top-1 accuracy, while by 2.3% and 4.1% on PASCAL VOC 2007 mAP, respectively.

Motivation & Objective

• Address performance degradation in compact CNNs caused by inter-group information loss in group and depthwise convolutions.
• Develop efficient architectural operations that enable feature fusion across channel groups without increasing computational cost.
• Design a new neural architecture, MENet, that maintains high accuracy under strict FLOP constraints for mobile deployment.
• Improve generalization in transfer learning scenarios, particularly for object detection tasks.
• Overcome limitations of existing methods like ShuffleNet, which suffer from diminishing returns when increasing channel groups.

Proposed method

• The merging operation aggregates features across all channels at the same spatial position into a narrow feature map, encoding inter-group information.
• The evolution operation applies depthwise and pointwise convolutions to extract spatial patterns from the merged narrow feature map.
• The ME module integrates merging and evolution into a single residual-like block, using depthwise and group convolutions for computational efficiency.
• Feature fusion is performed via element-wise product (scaling) rather than addition, enabling more effective inter-group representation learning.
• ME modules are stacked into a hierarchical network architecture, with increasing channel numbers in later stages to enhance representation capacity.
• Pre-trained MENet models are fine-tuned for transfer learning on object detection using Faster R-CNN with RoI align.

Experimental results

Research questions

• RQ1 How can inter-group information be effectively recovered in compact CNNs that rely on group and depthwise convolutions?
• RQ2 What is the impact of replacing element-wise addition with element-wise product in inter-group feature fusion?
• RQ3 How does the proposed ME module compare to existing modules like ShuffleNet’s channel shuffle in terms of accuracy and efficiency?
• RQ4 Can MENet generalize well to downstream tasks such as object detection under low FLOP budgets?
• RQ5 Does increasing the number of channel groups in MENet lead to consistent performance gains, unlike in ShuffleNet?

Key findings

• At 140 MFLOPs, MENet achieves 65.9% top-1 accuracy on ILSVRC 2012, outperforming ShuffleNet by 1% and MobileNet by 1.95%.
• On PASCAL VOC 2007 detection, MENet achieves 58.9% mAP at 140 MFLOPs, surpassing ShuffleNet (56.6%) by 2.3% and MobileNet (54.8%) by 4.1%.
• For the challenging 'boat' class, MENet improves mAP by 6.9% over ShuffleNet and 4.2% over MobileNet at 140 MFLOPs.
• At 550 MFLOPs, MENet exceeds ShuffleNet by 2% and MobileNet by 3.1% on mAP, demonstrating consistent gains across budgets.
• Using element-wise product for fusion yields 1.05% higher accuracy than element-wise addition when group number is 4, indicating superior representation learning.
• MENet maintains performance gains even with increased channel groups, unlike ShuffleNet, which degrades under the same conditions.