[Paper Review] Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks

Jie Hu, Li Shen+3 more|arXiv (Cornell University)|Oct 29, 2018

Advanced Neural Network Applications29 references386 citations

TL;DR

The paper introduces Gather-Excite (GE) modules that harvest broad feature context and reweight feature maps in CNNs, delivering noticeable accuracy gains with minimal parameter overhead across ImageNet, CIFAR, and object detection tasks, and demonstrating complementary gains to deeper networks.

ABSTRACT

While the use of bottom-up local operators in convolutional neural networks (CNNs) matches well some of the statistics of natural images, it may also prevent such models from capturing contextual long-range feature interactions. In this work, we propose a simple, lightweight approach for better context exploitation in CNNs. We do so by introducing a pair of operators: gather, which efficiently aggregates feature responses from a large spatial extent, and excite, which redistributes the pooled information to local features. The operators are cheap, both in terms of number of added parameters and computational complexity, and can be integrated directly in existing architectures to improve their performance. Experiments on several datasets show that gather-excite can bring benefits comparable to increasing the depth of a CNN at a fraction of the cost. For example, we find ResNet-50 with gather-excite operators is able to outperform its 101-layer counterpart on ImageNet with no additional learnable parameters. We also propose a parametric gather-excite operator pair which yields further performance gains, relate it to the recently-introduced Squeeze-and-Excitation Networks, and analyse the effects of these changes to the CNN feature activation statistics.

Motivation & Objective

• Motivate the need to improve CNN context exploitation beyond local receptive fields.
• Propose a lightweight GE framework composed of a gather operator and an excite operator.
• Demonstrate parameter-free and parameterized GE variants across architectures and datasets.
• Analyze how GE affects learned representations, convergence, and feature re-use.

Proposed method

• Define the GE framework with a gather operator that aggregates feature responses over large spatial extents.
• Define an excite operator that modulates input features conditioned on gathered context.
• Instantiate parameter-free GE (GE-θ−) using average pooling and interpolation, and parameterized GE (GE-θ, GE-θ+) using depth-wise convolutions and a channel subnetwork for excitation.
• Incorporate GE blocks into ResNet-50/101 and other backbones at various stages to assess performance, efficiency, and stage-wise impact.
• Compare GE variants against Squeeze-and-Excitation (SE) networks and deeper baselines across ImageNet, CIFAR-10/100, and MS COCO object detection.

Experimental results

Research questions

• RQ1 Does incorporating simple, context-aggregating GE modules improve CNN performance beyond local feature decisions?
• RQ2 How do parameter-free and parameterized GE variants compare in accuracy, compute, and parameter count?
• RQ3 Where in the network (which stages) do GE modules provide the most benefit?
• RQ4 How does GE interact with depth, architecture, and dataset scale (ImageNet, CIFAR, COCO)?

Key findings

• GE with global extent (GE-θ−) improves ResNet-50 from 23.30% to 22.14% top-1 error on ImageNet, approaching ResNet-101 performance.
• Parameterized GE (GE-θ) yields further gains over GE-θ−, e.g., 22.00% top-1 vs 23.30% baseline on ResNet-50, with modest FLOPs increase.
• Combining GE-θ+ (enhanced excitation) with GE-θ achieves 21.88% top-1 on ImageNet (ResNet-50), rivaling a 152-layer ResNet in accuracy with lower complexity.
• GE variants provide gains that transfer to deeper backbones (GE-θ−/GE-θ/GE-θ+) for ResNet-101, showing complementary improvements to depth.
• On ShuffleNet and other resource-constrained architectures, GE-θ+ delivers strong gains (e.g., CIFAR-10/100 improvements) under limited parameter budgets.
• In object detection (MS COCO), replacing ResNet-50 with GE-θ backbone improves mAP from 27.3 to 28.6.