[Paper Review] Second-order Non-local Attention Networks for Person Re-identification

Bryan Bryan, Xia+3 more|arXiv (Cornell University)|Aug 31, 2019

Video Surveillance and Tracking Methods48 references106 citations

TL;DR

Introduces a Second-order Non-local Attention (SONA) module to model long-range feature map correlations for person re-identification, combined with a generalized DropBlock, achieving state-of-the-art results on Market1501, CUHK03, and DukeMTMC-reID.

ABSTRACT

Recent efforts have shown promising results for person re-identification by designing part-based architectures to allow a neural network to learn discriminative representations from semantically coherent parts. Some efforts use soft attention to reallocate distant outliers to their most similar parts, while others adjust part granularity to incorporate more distant positions for learning the relationships. Others seek to generalize part-based methods by introducing a dropout mechanism on consecutive regions of the feature map to enhance distant region relationships. However, only few prior efforts model the distant or non-local positions of the feature map directly for the person re-ID task. In this paper, we propose a novel attention mechanism to directly model long-range relationships via second-order feature statistics. When combined with a generalized DropBlock module, our method performs equally to or better than state-of-the-art results for mainstream person re-identification datasets, including Market1501, CUHK03, and DukeMTMC-reID.

Motivation & Objective

• Motivate robust person re-ID without heavy reliance on rigid part partitions.
• Propose a second-order non-local attention mechanism to capture long-range feature map correlations.
• Enhance regularization with a generalized DropBlock to encourage learning distant relationships.
• Modify backbone with dilated convolutions to provide a larger spatial view for attention.
• Demonstrate state-of-the-art performance on major re-ID datasets and analyze component contributions.

Proposed method

• Present a backbone-branch architecture inspired by BFE with a global and a local branch.
• Inject the Second-order Non-local Attention (SONA) module after early ResNet stages to capture non-local correlations.
• Use a covariance-based attention map computed from reduced-dimension embeddings (theta and g) to form attention.
• Apply DropBlock+ with variable block sizes to encourage learning of diverse spatial relations.
• Dilate specific ResNet stages to enlarge feature map and provide broader spatial context for attention.
• Train with batch hard triplet loss and label-smoothed cross-entropy loss on respective branches.

Experimental results

Research questions

• RQ1 Can second-order statistics-based non-local attention effectively model long-range, cross-part relationships for person re-ID?
• RQ2 Does generalizing DropBlock to variable-sized blocks and dilating backbone stages improve performance and robustness across datasets?
• RQ3 Where is SONA best positioned within the backbone for optimal gains without harming efficiency?
• RQ4 How does SONA-Net compare to state-of-the-art part-based and non-part-based methods on Market1501, CUHK03, and DukeMTMC-reID?

Key findings

• SONA-Net achieves competitive and often superior results relative to state-of-the-art methods across Market1501, CUHK03, and DukeMTMC-reID.
• In ablations, adding SONA provides consistent gains; combining it with DropBlock+ yields the best performance.
• Placing SONA after early stages is effective; placing it after late stages degrades performance.
• Inference overhead from SONA is negligible (8.44 ms with SONA vs 7.89 ms without).
• Models with SONA variants (2-Net, 3-Net, 2+3-Net) show strong and stable improvements across datasets.