[Paper Review] Squeeze-and-Excitation Networks

Jie Hu, Li Shen+3 more|arXiv (Cornell University)|Sep 5, 2017

Advanced Neural Network Applications2,226 citations

TL;DR

Introduces Squeeze-and-Excitation (SE) blocks that recalibrate channel-wise feature responses to model interdependencies across channels, yielding significant accuracy gains with minimal computational overhead across various CNN architectures and datasets.

ABSTRACT

The central building block of convolutional neural networks (CNNs) is the convolution operator, which enables networks to construct informative features by fusing both spatial and channel-wise information within local receptive fields at each layer. A broad range of prior research has investigated the spatial component of this relationship, seeking to strengthen the representational power of a CNN by enhancing the quality of spatial encodings throughout its feature hierarchy. In this work, we focus instead on the channel relationship and propose a novel architectural unit, which we term the "Squeeze-and-Excitation" (SE) block, that adaptively recalibrates channel-wise feature responses by explicitly modelling interdependencies between channels. We show that these blocks can be stacked together to form SENet architectures that generalise extremely effectively across different datasets. We further demonstrate that SE blocks bring significant improvements in performance for existing state-of-the-art CNNs at slight additional computational cost. Squeeze-and-Excitation Networks formed the foundation of our ILSVRC 2017 classification submission which won first place and reduced the top-5 error to 2.251%, surpassing the winning entry of 2016 by a relative improvement of ~25%. Models and code are available at https://github.com/hujie-frank/SENet.

Motivation & Objective

• Investigate whether explicitly modeling interdependencies between channels can improve CNN representations.
• Develop a lightweight architectural unit (SE block) to recalibrate channel-wise feature responses.
• Demonstrate that SE blocks can be integrated as drop-in replacements across diverse architectures to improve performance without substantial compute cost.

Proposed method

• Define SE block consisting of a squeeze operation via global average pooling to capture global channel statistics.
• Design an excitation mechanism with a bottleneck of two fully-connected layers and a sigmoid gate to produce per-channel weights.
• Apply the learned per-channel weights to scale the corresponding feature maps (channel-wise reweighting).
• Stack SE blocks to form SENet architectures or replace blocks in existing networks (e.g., SE-ResNet, SE-Inception) for various depths and architectures.
• Evaluate SE blocks on ImageNet and additional datasets (CIFAR-10/100, Places365) and across architectures (ResNet, Inception, ResNeXt, MobileNet, ShuffleNet).
• Compare against baselines and report improvements with modest GFLOP overhead (r = 16 default).

Experimental results

Research questions

• RQ1 Do SE blocks improve classification accuracy across standard CNN architectures without large increases in computational cost?
• RQ2 Can channel-wise recalibration via SE blocks generalize across datasets and tasks beyond ImageNet?
• RQ3 What is the impact of SE blocks on different network depths and architectures?

Key findings

• SE blocks consistently improve top-1 and top-5 accuracy across ResNet, Inception-ResNet, ResNeXt, VGG-like, MobileNet and ShuffleNet baselines.
• SE-ResNet-50 achieves 6.62% top-5 error on ImageNet (0.86% improvement over ResNet-50) and closely approaches deeper ResNet-101 performance with roughly half the FLOPs.
• SE-ResNet-101 and SE-ResNet-152 outperform their non-SE counterparts and even some deeper models, with enhanced accuracy at similar or modestly increased computational cost.
• SE blocks provide gains on CIFAR-10 and CIFAR-100 across ResNet, WRN, and Shake-Shake variants, indicating robustness across small-scale datasets.
• On Places365, SE-ResNet-152 achieves lower top-5 error than ResNet-152 (11.01% vs 11.61%).
• In object detection with Faster R-CNN on COCO, SE-ResNet backbones improve AP@IoU=0.5 and AP compared to baseline ResNets (e.g., SE-ResNet-50 outperforms ResNet-50 by 2.4 AP points).