[Paper Review] Sharp Attention Network via Adaptive Sampling for Person Re-identification

Chen Shen, Guo-Jun Qi+5 more|arXiv (Cornell University)|May 7, 2018

Video Surveillance and Tracking Methods3 citations

TL;DR

This paper proposes a sharp attention network using adaptive sampling to generate more assertive, discrete attention masks for person re-identification (re-ID), replacing soft gating mechanisms with differentiable Gumbel-Softmax sampling. The method achieves state-of-the-art performance on CUHK03, Market-1501, and DukeMTMC-reID, with 88.3% rank-1 accuracy on CUHK03 labeled and 84.3% on detected, outperforming prior soft attention and SOTA methods.

ABSTRACT

In this paper, we present novel sharp attention networks by adaptively sampling feature maps from convolutional neural networks (CNNs) for person re-identification (re-ID) problem. Due to the introduction of sampling-based attention models, the proposed approach can adaptively generate sharper attention-aware feature masks. This greatly differs from the gating-based attention mechanism that relies soft gating functions to select the relevant features for person re-ID. In contrast, the proposed sampling-based attention mechanism allows us to effectively trim irrelevant features by enforcing the resultant feature masks to focus on the most discriminative features. It can produce sharper attentions that are more assertive in localizing subtle features relevant to re-identifying people across cameras. For this purpose, a differentiable Gumbel-Softmax sampler is employed to approximate the Bernoulli sampling to train the sharp attention networks. Extensive experimental evaluations demonstrate the superiority of this new sharp attention model for person re-ID over the other state-of-the-art methods on three challenging benchmarks including CUHK03, Market-1501, and DukeMTMC-reID.

Motivation & Objective

• To address the ambiguity in soft attention masks that struggle to localize subtle, discriminative features in person re-identification.
• To improve feature localization by generating sharper, more assertive attention masks that are either 1 (attended) or 0 (unattended), reducing uncertainty.
• To enable end-to-end training of sharp attention mechanisms via differentiable Gumbel-Softmax sampling, overcoming the non-differentiability of discrete sampling.
• To enhance performance in complex re-ID scenarios by integrating a context-aware front-end unit to guide attention sampling.
• To outperform existing soft-gating attention models and state-of-the-art re-ID methods on multiple benchmarks.

Proposed method

• The method employs a differentiable Gumbel-Softmax sampler to approximate Bernoulli sampling, enabling backpropagation through discrete attention masks.
• Sharp attention masks are generated by sampling directly from convolutional feature maps, resulting in binary-like masks with values close to 0 or 1, minimizing attention ambiguity.
• A cross-feature interaction learning scheme is introduced to jointly optimize original CNN features and sharp attention features for improved representation.
• A context-aware front-end unit is added to the attention mask generator to better guide sampling in complex visual contexts, especially where discriminative cues are context-dependent.
• The model is trained end-to-end using standard re-ID loss functions, with the Gumbel-Softmax relaxation allowing gradient flow through the sampling operation.
• The attention mask generator is trained to produce masks that focus on discriminative parts (e.g., distinctive clothing items) to improve cross-camera person matching.

Experimental results

Research questions

• RQ1 Can a sampling-based attention mechanism produce sharper, more assertive attention masks than soft-gating mechanisms in person re-identification?
• RQ2 Does the use of differentiable Gumbel-Softmax sampling enable effective end-to-end training of discrete attention masks in deep CNNs?
• RQ3 Can sharp attention masks improve person re-identification accuracy by better localizing subtle, discriminative features across cameras?
• RQ4 How does the proposed method compare to state-of-the-art re-identification models on standard benchmarks like Market-1501, CUHK03, and DukeMTMC-reID?
• RQ5 Does integrating a context-aware front-end unit enhance the performance of sharp attention in complex or ambiguous visual scenarios?

Key findings

• On the CUHK03 labeled dataset, the proposed method achieves 88.3% rank-1 accuracy, 98.8% rank-5, and 99.4% rank-10 accuracy, with 85.9% mAP, outperforming all prior state-of-the-art methods.
• On the CUHK03 detected dataset, the method achieves 84.3% rank-1 accuracy and 82.2% mAP, setting a new SOTA on this challenging benchmark.
• On Market-1501, the method achieves 85.9% rank-1 accuracy and 70.1% mAP, surpassing the second-best method (PDC) by 1.5% in rank-1 and 6.7% in mAP.
• On DukeMTMC-reID, the method achieves 77.9% rank-1 accuracy and 58.8% mAP, setting a new state-of-the-art on this dataset.
• The ablation study confirms that the sharp attention mechanism significantly improves performance over soft attention baselines, especially in localizing fine-grained visual cues like distinctive bags or clothing patterns.
• The integration of the context-aware front-end unit further boosts performance, demonstrating its effectiveness in guiding attention in complex scenes.