[Paper Review] Self Attention Grid for Person Re-Identification

Jean-Paul Ainam, Ke Qin+1 more|arXiv (Cornell University)|Sep 23, 2018

Video Surveillance and Tracking Methods28 references3 citations

TL;DR

This paper proposes Self Attention Grid (SAG), a novel self-attention mechanism for person re-identification that enhances feature learning by focusing on multiple discriminative regions in high-resolution feature maps. By using two parallel branches—one processing high-res features and the other learning a filtering attention grid via max-pooling and softmax—SAG improves accuracy on three benchmark datasets, achieving state-of-the-art performance, including 81.05% mAP on DukeMTMC-ReID with a residual re-ranking (RR) strategy.

ABSTRACT

In this paper, we present an attention mechanism scheme to improve person re-identification task. Inspired by biology, we propose Self Attention Grid (SAG) to discover the most informative parts from a high-resolution image using its internal representation. In particular, given an input image, the proposed model is fed with two copies of the same image and consists of two branches. The upper branch processes the high-resolution image and learns high dimensional feature representation while the lower branch processes the low-resolution image and learn a filtering attention grid. We apply a max filter operation to non-overlapping sub-regions on the high feature representation before element-wise multiplied with the output of the second branch. The feature maps of the second branch are subsequently weighted to reflect the importance of each patch of the grid using a softmax operation. Our attention module helps the network learn the most discriminative visual features of multiple image regions and is specifically optimized to attend feature representation at different levels. Extensive experiments on three large-scale datasets show that our self-attention mechanism significantly improves the baseline model and outperforms various state-of-art models by a large margin.

Motivation & Objective

• To address the challenge of low accuracy in person re-identification under variable lighting, pose, and low-resolution conditions.
• To improve feature discrimination by enabling the network to focus on multiple informative image regions simultaneously, rather than a single region.
• To design a plug-and-play, fully differentiable attention module compatible with any CNN architecture.
• To preserve both high-level discriminative features and low-level spatial details through a dual-branch attention mechanism.
• To achieve state-of-the-art performance on large-scale person re-ID benchmarks without architectural overhauls.

Proposed method

• The model uses two parallel branches: one processes the high-resolution feature map, and the other learns an attention grid from a low-resolution version of the same input.
• The attention grid is generated via max-pooling over non-overlapping spatial regions followed by a softmax operation to produce attention weights.
• The high-resolution features are then element-wise multiplied by the attention grid to emphasize important regions and suppress background noise.
• The attention mechanism is applied at multiple depth levels (D1 to D4), allowing multi-level feature refinement.
• The entire module is differentiable and trained end-to-end using backpropagation with standard optimization.
• L2 normalization is applied selectively to upper residual blocks to stabilize training and improve feature quality.

Experimental results

Research questions

• RQ1 Can a self-attention mechanism that focuses on multiple non-overlapping image regions improve person re-identification accuracy?
• RQ2 Does a dual-branch architecture, combining high-res features with a learned attention grid, enhance feature discrimination without architectural complexity?
• RQ3 How does the proposed SAG module compare to existing attention mechanisms in terms of performance and generalization across datasets?
• RQ4 What is the optimal placement and configuration of the SAG module within a CNN for maximum performance gain?
• RQ5 Can the SAG module be effectively combined with post-processing techniques like residual re-ranking to further boost accuracy?

Key findings

• On DukeMTMC-ReID, SAG achieved 79.94% R1 and 60.88% mAP, outperforming most state-of-the-art methods except DCC, which was slightly better by 0.36% in R1.
• With residual re-ranking (SAG+RR), the model achieved 85.28% R1 and 81.05% mAP, significantly surpassing the baseline and most prior methods.
• The ablation study confirmed that SAG generalizes well across deep layers, with performance gains most evident in higher-level features.
• The visualization of attention grids showed that SAG effectively highlights discriminative parts such as faces, backpacks, and clothing, while suppressing background regions.
• Applying L2 normalization only to the first three residual blocks improved accuracy, but applying it to the fourth block degraded performance, indicating layer-specific sensitivity.
• The SAG module is modular and can be easily integrated into existing CNN architectures without architectural changes, demonstrating strong practicality.