[Paper Review] Coordinate Attention for Efficient Mobile Network Design

Qibin Hou, Daquan Zhou+1 more|arXiv (Cornell University)|Mar 4, 2021

Advanced Neural Network Applications49 references321 citations

TL;DR

The paper introduces coordinate attention, a lightweight attention module for mobile networks that embeds positional information by factorizing 2D pooling into two 1D pools, improving ImageNet classification and downstream vision tasks with minimal overhead.

ABSTRACT

Recent studies on mobile network design have demonstrated the remarkable effectiveness of channel attention (e.g., the Squeeze-and-Excitation attention) for lifting model performance, but they generally neglect the positional information, which is important for generating spatially selective attention maps. In this paper, we propose a novel attention mechanism for mobile networks by embedding positional information into channel attention, which we call "coordinate attention". Unlike channel attention that transforms a feature tensor to a single feature vector via 2D global pooling, the coordinate attention factorizes channel attention into two 1D feature encoding processes that aggregate features along the two spatial directions, respectively. In this way, long-range dependencies can be captured along one spatial direction and meanwhile precise positional information can be preserved along the other spatial direction. The resulting feature maps are then encoded separately into a pair of direction-aware and position-sensitive attention maps that can be complementarily applied to the input feature map to augment the representations of the objects of interest. Our coordinate attention is simple and can be flexibly plugged into classic mobile networks, such as MobileNetV2, MobileNeXt, and EfficientNet with nearly no computational overhead. Extensive experiments demonstrate that our coordinate attention is not only beneficial to ImageNet classification but more interestingly, behaves better in down-stream tasks, such as object detection and semantic segmentation. Code is available at https://github.com/Andrew-Qibin/CoordAttention.

Motivation & Objective

• Motivate the need for attention mechanisms that preserve spatial (positional) information in mobile networks.
• Propose a new attention block that embedding coordinate information while keeping computation low.
• Show that coordinate attention can be plugged into existing mobile building blocks (e.g., MobileNetV2, MobileNeXt, EfficientNet) with minimal overhead.
• Demonstrate improvements on ImageNet classification and downstream tasks such as object detection and semantic segmentation.

Proposed method

• Factorize channel attention into two parallel 1D feature encoding processes that pool along horizontal and vertical directions.
• Concatenate the two 1D pooled features and pass through shared 1x1 convolutions to generate direction-aware attention maps (g^h and g^w).
• Apply the attention maps to the input feature map via element-wise multiplication: Y_c(i,j) = X_c(i,j) * g^h_c(i) * g^w_c(j).
• Use a reduction ratio r to control bottleneck size and keep computation light for mobile settings.
• Demonstrate plug-in compatibility with inverted residual blocks (MobileNetV2) and sandglass bottlenecks (MobileNeXt), and evaluate on ImageNet, COCO, VOC, and Cityscapes.

Figure 1: Performance of different attention methods on three classic vision tasks. The y-axis labels from left to right are top-1 accuracy, mean IoU, and AP, respectively. Clearly, our approach not only achieves the best result in ImageNet classification [ 33 ] against the SE block [ 18 ] and CBAM

Experimental results

Research questions

• RQ1 Does embedding coordinate information via two 1D poolings improve performance over SE and CBAM in mobile networks?
• RQ2 How does coordinate attention perform when plugged into different mobile backbones (e.g., MobileNetV2, MobileNeXt, EfficientNet) compared to baseline attention methods?
• RQ3 What is the impact of the reduction ratio on accuracy and model size, and is coordinate attention robust to this hyperparameter?
• RQ4 Do coordinate-attention-equipped models transfer better to downstream tasks like object detection and semantic segmentation than other lightweight attentions?

Key findings

• Coordinate attention yields superior performance over SE and CBAM on mobile networks for ImageNet classification.
• Embedding spatial coordinate information via two 1D poolings preserves long-range dependencies with positional cues, improving localization of objects of interest.
• Across tested backbones (MobileNetV2, MobileNeXt, EfficientNet), coordinate attention provides consistent gains with minimal overhead, and shows notable improvements on downstream tasks like object detection and semantic segmentation.
• Ablation shows that combining horizontal and vertical attention is more effective than using either alone, confirming the value of coordinate information embedding.
• Visualizations indicate that coordinate attention better highlights objects of interest in feature maps compared to SE and CBAM.

Figure 2: Schematic comparison of the proposed coordinate attention block (c) to the classic SE channel attention block [ 18 ] (a) and CBAM [ 44 ] (b). Here, “GAP” and “GMP” refer to the global average pooling and global max pooling, respectively. ‘X Avg Pool’ and ’Y Avg Pool’ refer to 1D horizontal