[Paper Review] Dual Attention Matching Network for Context-Aware Feature Sequence based Person Re-Identification

Jianlou Si, Honggang Zhang+5 more|arXiv (Cornell University)|Mar 27, 2018

Video Surveillance and Tracking MethodsComputer Science32 references76 citations

TL;DR

DuATM introduces a dual attention mechanism to refine and align context-aware feature sequences for image and video person re-identification, outperforming prior attention and sequence-based methods. It uses a siamese/triplet training setup with auxiliary losses and yields state-of-the-art results on Market-1501, DukeMTMC-reID, and MARS.

ABSTRACT

Typical person re-identification (ReID) methods usually describe each pedestrian with a single feature vector and match them in a task-specific metric space. However, the methods based on a single feature vector are not sufficient enough to overcome visual ambiguity, which frequently occurs in real scenario. In this paper, we propose a novel end-to-end trainable framework, called Dual ATtention Matching network (DuATM), to learn context-aware feature sequences and perform attentive sequence comparison simultaneously. The core component of our DuATM framework is a dual attention mechanism, in which both intra-sequence and inter-sequence attention strategies are used for feature refinement and feature-pair alignment, respectively. Thus, detailed visual cues contained in the intermediate feature sequences can be automatically exploited and properly compared. We train the proposed DuATM network as a siamese network via a triplet loss assisted with a de-correlation loss and a cross-entropy loss. We conduct extensive experiments on both image and video based ReID benchmark datasets. Experimental results demonstrate the significant advantages of our approach compared to the state-of-the-art methods.

Motivation & Objective

• Motivate the limitations of single-feature-vector ReID representations under appearance variations and occlusions.
• Propose an end-to-end framework that learns context-aware feature sequences and performs attentive sequence matching.
• Introduce a dual attention mechanism to refine intra-sequence features and align inter-sequence pairs.
• Evaluate on image and video ReID benchmarks to demonstrate performance gains over state-of-the-art.

Proposed method

• Extract feature sequences from images/videos using DenseNet-121 backbones (image) or DenseNet-121 plus bidirectional temporal modeling (video).
• Apply a dual attention block to perform intra-sequence refinement and inter-sequence alignment for each feature in a pair.
• Compute pairwise distances between refined and aligned features and aggregate into a holistic sequence distance.
• Train the network in a siamese setup with triplet loss, and add de-correlation and cross-entropy auxiliary losses to improve compactness and discriminability.
• Use data augmentation and hard triplet mining to address training data imbalance.
• Optimize with SGD, fine-tune DenseNet, and evaluate with CMC and mAP metrics.

Experimental results

Research questions

• RQ1 Can dual intra-sequence refinement and inter-sequence alignment improve robustness to misalignment and corruptions in feature sequences for ReID?
• RQ2 Do auxiliary losses (de-correlation and cross-entropy) provide tangible improvements for context-aware sequence representations?
• RQ3 How does DuATM perform on both image- and video-based ReID benchmarks compared to state-of-the-art methods?
• RQ4 What are the effects of sequence length, feature dimension, and training parameters on performance?

Key findings

• DuATM with both intra- and inter-sequence attention outperforms the AvePool+ baseline across Market-1501, DukeMTMC-reID, and MARS.
• Adding de-correlation and cross-entropy losses yields further gains, with cross-entropy providing substantial improvements due to identity supervision.
• Ablation shows dual attention (both intra- and inter-attention) beats using a single attention type.
• DuATM achieves top performance on Market-1501 (R1 91.42, mAP 76.62), DukeMTMC-reID (R1 81.82, mAP 64.58), and MARS (R1 78.74, mAP 62.26).
• Longer video sequence length T improves mAP significantly on MARS (e.g., mAP from 21.87% at T=1 to 59.42% at T=96).
• Visualization indicates intra-sequence attention focuses on contextual body parts while inter-sequence attention aligns semantically consistent parts across sequences.