[Paper Review] Dynamic Region-Aware Convolution

Jin Chen, Xijun Wang+3 more|arXiv (Cornell University)|Mar 27, 2020

Advanced Image and Video Retrieval Techniques37 references9 citations

TL;DR

This paper proposes Dynamic Region-Aware Convolution (DRConv), a novel convolutional operation that dynamically assigns spatially shared filters to semantic regions via a learnable guided mask, enabling improved feature representation while preserving translation-invariance and computational efficiency. DRConv achieves state-of-the-art performance on ImageNet with 67.1% top-1 accuracy at 46M FLOPs, showing a 6.3% relative improvement over standard convolution.

ABSTRACT

We propose a new convolution called Dynamic Region-Aware Convolution (DRConv), which can automatically assign multiple filters to corresponding spatial regions where features have similar representation. In this way, DRConv outperforms standard convolution in modeling semantic variations. Standard convolutional layer can increase the number of filers to extract more visual elements but results in high computational cost. More gracefully, our DRConv transfers the increasing channel-wise filters to spatial dimension with learnable instructor, which not only improve representation ability of convolution, but also maintains computational cost and the translation-invariance as standard convolution dose. DRConv is an effective and elegant method for handling complex and variable spatial information distribution. It can substitute standard convolution in any existing networks for its plug-and-play property, especially to power convolution layers in efficient networks. We evaluate DRConv on a wide range of models (MobileNet series, ShuffleNetV2, etc.) and tasks (Classification, Face Recognition, Detection and Segmentation). On ImageNet classification, DRConv-based ShuffleNetV2-0.5x achieves state-of-the-art performance of 67.1% at 46M multiply-adds level with 6.3% relative improvement.

Motivation & Objective

• To address the limitations of standard convolution in modeling spatial semantic variations without increasing computational cost.
• To overcome the parameter explosion and loss of translation-invariance in local convolution methods that assign unique filters per spatial location.
• To enable sample-specific, dynamic filter assignment across spatial regions using a learnable guided mask for improved representation.
• To design a plug-and-play module that enhances existing networks without architectural overhaul.
• To maintain computational efficiency while significantly improving performance on diverse vision tasks.

Proposed method

• DRConv uses a standard convolution to generate a guided feature map from input, which is then used to predict a learnable guided mask that partitions the spatial dimension into m regions.
• The guided mask determines region-sharing patterns, where each region shares a single filter, and filters are dynamically generated per region via a filter generator module.
• Each region applies a unique, sample-specific filter to its spatial region, enabling localized feature modeling while preserving translation-invariance.
• The filter generator is a lightweight, learnable module whose parameters are independent of spatial size, minimizing parameter growth.
• Backward propagation is specially designed to optimize the guided mask based on task loss, enabling end-to-end training.
• The method is plug-and-play and can replace standard convolutions in any network, including efficient architectures like MobileNet and ShuffleNetV2.

Experimental results

Research questions

• RQ1 Can dynamic, region-aware filter assignment improve feature representation in convolutional networks without increasing computational cost?
• RQ2 How does the learnable guided mask influence the spatial distribution of filters and performance across different tasks?
• RQ3 To what extent does DRConv improve performance on small and efficient models compared to standard convolution?
• RQ4 Does DRConv maintain translation-invariance while enabling spatially adaptive filtering, unlike local convolution methods?
• RQ5 What is the optimal number of regions for the guided mask in terms of performance and efficiency?

Key findings

• DRConv-based ShuffleNetV2-0.5× achieves 67.1% top-1 accuracy on ImageNet at 46M FLOPs, representing a 6.3% relative improvement over standard convolution.
• On COCO detection, replacing two FPN layers with DRConv (16 regions) improves AP by 1.2%, and with 8 regions in DetNAS-300M, it improves AP by 1.8%.
• On COCO instance segmentation, DRConv improves AP by 1.1% when used in two FPN layers with 16 regions.
• Smaller models like MobileNetV2-0.25× and ShuffleNetV2-0.5× benefit most from DRConv, showing greater relative performance gains than larger models.
• Visualization confirms that the guided mask learns semantically coherent regions, with deeper layers forming connected regions and shallower layers having discrete, context-sensitive regions.
• Ablation studies show that increasing the number of regions improves performance, indicating that finer spatial specialization enhances feature learning.