[Paper Review] Deep Attention Aware Feature Learning for Person Re-Identification

Yifan Chen, Han Wang+3 more|arXiv (Cornell University)|Mar 1, 2020

Video Surveillance and Tracking Methods4 citations

TL;DR

This paper proposes a lightweight, structure-agnostic attention mechanism for person re-identification by adding holistic (HAB) and partial (PAB) attention branches during training, without modifying the backbone network. The method enhances feature discrimination by focusing on the person (HAB) and body parts (PAB) via channel-wise decoupling, achieving state-of-the-art performance with no increase in inference time or model size.

ABSTRACT

Visual attention has proven to be effective in improving the performance of person re-identification. Most existing methods apply visual attention heuristically by learning an additional attention map to re-weight the feature maps for person re-identification. However, this kind of methods inevitably increase the model complexity and inference time. In this paper, we propose to incorporate the attention learning as additional objectives in a person ReID network without changing the original structure, thus maintain the same inference time and model size. Two kinds of attentions have been considered to make the learned feature maps being aware of the person and related body parts respectively. Globally, a holistic attention branch (HAB) makes the feature maps obtained by backbone focus on persons so as to alleviate the influence of background. Locally, a partial attention branch (PAB) makes the extracted features be decoupled into several groups and be separately responsible for different body parts (i.e., keypoints), thus increasing the robustness to pose variation and partial occlusion. These two kinds of attentions are universal and can be incorporated into existing ReID networks. We have tested its performance on two typical networks (TriNet and Bag of Tricks) and observed significant performance improvement on five widely used datasets.

Motivation & Objective

• To improve person re-identification performance by incorporating attention mechanisms without increasing model complexity or inference time.
• To address challenges such as background clutter, pose variation, and partial occlusion through implicit attention learning in feature maps.
• To enable attention learning via additional supervision during training, rather than adding extra network branches or layers.
• To explore channel-wise attention as a means of decoupling features for different body parts, enhancing robustness to occlusion and pose changes.
• To validate the method’s generalization across different backbone networks and benchmarks without architectural modifications.

Proposed method

• Introduces a holistic attention branch (HAB) that supervises the backbone to focus on the person by predicting a person mask, reducing background interference.
• Proposes a partial attention branch (PAB) that uses shared deconvolutional decoders to predict keypoint heatmaps, enforcing channel-wise feature decoupling for different body parts.
• Employs shared-weights deconvolution layers in PAB to ensure the decoder is not biased toward any specific body part, preserving attention specificity per channel group.
• Uses 1x1 convolutional layers to group feature channels and assign them to specific body parts, with ablation showing that feature maps—not 1x1 conv layers—determine output attention patterns.
• Jointly optimizes HAB and PAB with the base ReID network during training, using only additional supervision signals, not architectural changes.
• Applies the method to existing ReID networks (e.g., TriNet, Bag of Tricks) without modifying their structure, maintaining original inference speed and model size.

Experimental results

Research questions

• RQ1 Can attention mechanisms be effectively embedded in ReID feature learning without adding extra network parameters or increasing inference time?
• RQ2 How does jointly optimizing holistic and partial attention branches improve robustness to background clutter and pose variation?
• RQ3 To what extent does channel-wise decoupling of features via PAB enhance discriminative capacity under partial occlusion?
• RQ4 Does using shared decoders in PAB preserve attention specificity better than independent decoders for each body part?
• RQ5 How does the 1x1 convolution shuffle experiment validate the role of feature maps versus 1x1 conv layers in attention learning?

Key findings

• The proposed method achieves state-of-the-art performance on five widely used ReID benchmarks, including Market-1501 and DukeMTMC-reID, with mAP and Rank-1 accuracy improvements.
• On Market-1501, the method achieves 85.87% mAP and 96.12% Rank-1 accuracy when integrated with the Bag of Tricks backbone, outperforming previous methods.
• Using shared decoders in PAB yields better performance than independent decoders, with a +3.83% mAP gain for keypoint prediction and +2.74% for part image prediction.
• The 1x1 convolution shuffle experiment confirms that attention patterns are primarily determined by input feature maps, not the 1x1 conv layers, validating effective channel-wise decoupling.
• Ablation studies show that both HAB and PAB contribute significantly, with HAB reducing background interference and PAB improving robustness to occlusion and pose variation.
• The method maintains the same inference time and model size as the original network, proving effective for real-time deployment.