[Paper Review] BA-Net: Bridge Attention for Deep Convolutional Neural Networks

Yue Zhao, Junzhou Chen+2 more|arXiv (Cornell University)|Dec 8, 2021

Advanced Neural Network Applications4 citations

TL;DR

This paper proposes BA-Net, a novel attention mechanism that enhances deep CNNs by bridging features from all convolutional layers within each residual block using batch normalization and fully connected layers. By integrating multi-level features instead of relying solely on the immediate preceding layer, BA-Net achieves superior performance with minimal computational overhead, outperforming SENet by 0.71% top-1 accuracy on ImageNet using ResNet-50.

ABSTRACT

In recent years, channel attention mechanism has been widely investigated due to its great potential in improving the performance of deep convolutional neural networks (CNNs) in many vision tasks. However, in most of the existing methods, only the output of the adjacent convolution layer is fed into the attention layer for calculating the channel weights. Information from other convolution layers has been ignored. With these observations, a simple strategy, named Bridge Attention Net (BA-Net), is proposed in this paper for better performance with channel attention mechanisms. The core idea of this design is to bridge the outputs of the previous convolution layers through skip connections for channel weights generation. Based on our experiment and theory analysis, we find that features from previous layers also contribute to the weights significantly. The Comprehensive evaluation demonstrates that the proposed approach achieves state-of-the-art(SOTA) performance compared with the existing methods in accuracy and speed. which shows that Bridge Attention provides a new perspective on the design of neural network architectures with great potential in improving performance. The code is available at https://github.com/zhaoy376/Bridge-Attention.

Motivation & Objective

• To address the limitation of existing channel attention mechanisms that rely only on the immediate preceding convolution layer, leading to suboptimal information utilization.
• To explore whether integrating features from earlier layers within a residual block can improve attention weight estimation and model performance.
• To design a simple, efficient, and generalizable attention module that enhances existing CNN architectures without increasing model complexity or inference cost.
• To empirically evaluate the effectiveness of different types and distances of previous features in the attention mechanism.
• To provide a stable and easily deployable attention module that outperforms state-of-the-art methods across multiple vision tasks.

Proposed method

• Proposes the Bridge Attention Module (BAM) that concatenates feature maps from all three convolutional layers within a residual block before applying global average pooling and fully connected layers.
• Uses batch normalization (BN) after the fully connected layer to stabilize training and improve gradient flow.
• Introduces a simple, lightweight design using only FC and BN layers, avoiding complex operations like 1D convolutions or channel shuffling.
• Applies the BAM module after the third convolutional layer in each residual block, replacing the standard attention layer in models like SENet.
• Employs a residual learning framework where the BAM is plugged into existing architectures such as ResNet-50 with minimal architectural changes.
• Uses random forest models to analyze feature importance and quantify the contribution of each intermediate feature to the final attention weights.

Experimental results

Research questions

• RQ1 Can integrating features from multiple previous convolutional layers within a residual block improve channel attention performance compared to using only the last layer?
• RQ2 What is the optimal configuration of bridged features—specifically, which types and distances of previous features yield the best performance?
• RQ3 Does bridging all convolution outputs inside each block lead to more diverse and informative attention weights compared to standard attention mechanisms?
• RQ4 How does the contribution of earlier convolutional features compare to the immediate preceding layer in determining attention weights?
• RQ5 Can a simple, lightweight attention module based on FC and BN achieve state-of-the-art performance with low computational cost?

Key findings

• Bridging all convolution outputs within each residual block using FC and BN leads to the best performance, significantly outperforming methods that use only the last convolutional layer.
• BA-Net achieves a 0.71% higher top-1 accuracy than SENet on ImageNet using ResNet-50, with comparable model complexity and inference speed.
• The attention weight distribution in BA-Net shows higher variance in deeper stages, indicating better discrimination of important features across different classes.
• Visualizations show that BA-Net's attention weights are more distinguishable across classes than SENet’s, especially in later stages, suggesting improved feature representation.
• Random forest analysis reveals that features from earlier layers (e.g., second or first conv layers) contribute significantly to attention weights, sometimes even more than the immediate preceding layer.
• The method is stable and generalizable, showing consistent performance gains across image classification, object detection, and instance segmentation tasks.