[Paper Review] GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond

Yue Cao, Jiarui Xu+3 more|arXiv (Cornell University)|Apr 25, 2019

Advanced Neural Network ApplicationsComputer Science42 references213 citations

TL;DR

GCNet introduces a lightweight global context block that unifies simplified non-local and SE blocks into a three-step framework, yielding better long-range dependency modeling and improved performance across object detection/segmentation, image classification, and action recognition with minimal FLOP overhead.

ABSTRACT

The Non-Local Network (NLNet) presents a pioneering approach for capturing long-range dependencies, via aggregating query-specific global context to each query position. However, through a rigorous empirical analysis, we have found that the global contexts modeled by non-local network are almost the same for different query positions within an image. In this paper, we take advantage of this finding to create a simplified network based on a query-independent formulation, which maintains the accuracy of NLNet but with significantly less computation. We further observe that this simplified design shares similar structure with Squeeze-Excitation Network (SENet). Hence we unify them into a three-step general framework for global context modeling. Within the general framework, we design a better instantiation, called the global context (GC) block, which is lightweight and can effectively model the global context. The lightweight property allows us to apply it for multiple layers in a backbone network to construct a global context network (GCNet), which generally outperforms both simplified NLNet and SENet on major benchmarks for various recognition tasks. The code and configurations are released at https://github.com/xvjiarui/GCNet.

Motivation & Objective

• Motivate and understand long-range dependency modeling in CNNs and its computational costs.
• Unify simplified non-local and SE approaches into a general global context modeling framework.
• Design a lightweight GC block that captures global context efficiently and can be integrated across backbone networks.
• Empirically validate GCNet on COCO object detection/segmentation, ImageNet classification, and Kinetics action recognition, through ablations and backbone experiments.

Proposed method

• Observe that query-specific attention maps in NLNet are nearly identical across query positions, motivating a query-independent global context approach.
• Simplify the NL block by using a shared global attention map for all query positions and apply a bottleneck transform to reduce parameters (GC design).
• Abstract the approach into a three-step global context modeling framework: (a) global context modeling via attention pooling; (b) feature transform (bottleneck); (c) fusion via addition to all positions.
• Propose the GC block as a concrete instantiation of this framework, combining efficient global attention pooling with a bottleneck transform and addition-based fusion.
• Demonstrate how GCBlock can be inserted into multiple layers of ResNet/ResNeXt backbones with minimal FLOPs increase (GCNet backbone).
• Compare against NLNet and SENet across standard vision tasks to establish efficiency-accuracy tradeoffs.

Experimental results

Research questions

• RQ1 Can a query-independent global context provide similar usefulness to the query-specific NL attention with substantially reduced computation?
• RQ2 How does the GC block compare to NL and SE blocks in terms of accuracy gains and parameter/FLOP costs across different backbones and tasks?
• RQ3 What architectural choices (context modeling, transform, fusion) maximize performance while keeping computation lightweight?
• RQ4 Is GCNet beneficial when applied to multiple stages of a backbone (e.g., c3, c4, c5 in ResNet) and across varied datasets/tasks (COCO, ImageNet, Kinetics)?

Key findings

• GCNet consistently outperforms NLNet and SENet on major benchmarks with similar or slightly higher FLOPs (e.g., COCO AP improvements of about 1.5–1.9 points for box and mask).
• GCNet achieves about 0.07% relative FLOP increase while gaining notable accuracy on COCO object detection/segmentation (AP^bbox and AP^mask).
• GCNet yields performance gains on ImageNet classification (≈0.8% top-1) and Kinetics action recognition (≈1.1% top-1) with minimal computation overhead.
• The GC block enables inserting into multiple layers (c3+c4+c5) with modest parameter/compute growth (e.g., ~2.5M extra parameters and ~0.26% FLOP increase for ResNet-50).
• Ablation studies show that addition fusion and bottleneck transform with layer normalization provide strong performance, and global attention pooling is effective though less critical than fusion choice.