[Paper Review] Relation-Aware Global Attention for Person Re-identification

Zhizheng Zhang, Cuiling Lan+3 more|arXiv (Cornell University)|Apr 5, 2019

Video Surveillance and Tracking MethodsComputer Science67 references38 citations

TL;DR

The paper introduces a Relation-Aware Global Attention (RGA) module that learns global structural relations for each feature node to produce spatial and channel attentions, achieving state-of-the-art re-id performance on CUHK03, Market1501, and MSMT17.

ABSTRACT

For person re-identification (re-id), attention mechanisms have become attractive as they aim at strengthening discriminative features and suppressing irrelevant ones, which matches well the key of re-id, i.e., discriminative feature learning. Previous approaches typically learn attention using local convolutions, ignoring the mining of knowledge from global structure patterns. Intuitively, the affinities among spatial positions/nodes in the feature map provide clustering-like information and are helpful for inferring semantics and thus attention, especially for person images where the feasible human poses are constrained. In this work, we propose an effective Relation-Aware Global Attention (RGA) module which captures the global structural information for better attention learning. Specifically, for each feature position, in order to compactly grasp the structural information of global scope and local appearance information, we propose to stack the relations, i.e., its pairwise correlations/affinities with all the feature positions (e.g., in raster scan order), and the feature itself together to learn the attention with a shallow convolutional model. Extensive ablation studies demonstrate that our RGA can significantly enhance the feature representation power and help achieve the state-of-the-art performance on several popular benchmarks. The source code is available at https://github.com/microsoft/Relation-Aware-Global-Attention-Networks.

Motivation & Objective

• Motivate attention learning for person re-id to exploit global structural information beyond local receptive fields.
• Propose a compact mechanism to mine semantics from global relations for each feature node.
• Develop spatial (RGA-S) and channel (RGA-C) relation-aware global attention modules and demonstrate their effectiveness.
• Show that combining RGA-S and RGA-C yields state-of-the-art results on key re-id benchmarks.

Proposed method

• Model pairwise relations (affinities) between feature nodes and stack them to form a global relation vector for each node.
• For Spatial RGA (RGA-S): compute r_i,j = f_s(x_i, x_j) via embedded 1x1 convolutions, form r_i = [R_s(i,:), R_s(:,i)], and combine with x_i to predict attention a_i through a small two-layer convnet.
• For Channel RGA (RGA-C): treat channels as nodes, compute r_i,j similarly with embedded features, form r_i, and derive channel attention a_i in the same fashion as spatial but along the channel dimension.
• Fuse local feature (via embedding) and global relation vector to produce a relation-aware feature and attention using a two-layer convnet with sigmoid output.
• Integrate RGA modules into a ResNet-50 backbone (RetNet-50 variant) and evaluate on CUHK03, Market1501, and MSMT17.

Experimental results

Research questions

• RQ1 Can global structure and pairwise relations among feature positions be exploited to improve attention for person re-id?
• RQ2 Do spatial and channel relation-aware attentions complement each other to yield better discriminative features?
• RQ3 How does RGA compare to existing attention mechanisms (local attention, non-local, CBAM) on standard re-id benchmarks?
• RQ4 What is the impact of embedding choices and module placement within the backbone on performance?

Key findings

• RGA-S and RGA-C both significantly improve performance over the baseline on CUHK03 and Market1501.
• Combining spatial and channel attentions (RGA-SC) yields the best results, outperforming baselines by up to 8.4% in mAP on CUHK03 and achieving strong gains on Market1501 and MSMT17.
• RGA-S and RGA-C outperform several attention baselines (CBAM, FC-S/FC-C, SE, NL) in both Rank-1 and mAP metrics.
• Asymmetric embedding for relation modeling provides further gains over symmetric or no embedding.
• RGA-SC achieves state-of-the-art results on CUHK03, Market1501, and MSMT17 among reported methods, with notable improvements over the second-best approaches.