[Paper Review] Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification

Mang Ye, Jianbing Shen+3 more|arXiv (Cornell University)|Jul 18, 2020

Video Surveillance and Tracking Methods61 references32 citations

TL;DR

The paper proposes DDAG, a two-stream VI-ReID framework combining intra-modality weighted-part aggregation and cross-modality graph structured attention, with a parameter-free dynamic dual aggregation strategy that progressively integrates both components during training.

ABSTRACT

Visible-infrared person re-identification (VI-ReID) is a challenging cross-modality pedestrian retrieval problem. Due to the large intra-class variations and cross-modality discrepancy with large amount of sample noise, it is difficult to learn discriminative part features. Existing VI-ReID methods instead tend to learn global representations, which have limited discriminability and weak robustness to noisy images. In this paper, we propose a novel dynamic dual-attentive aggregation (DDAG) learning method by mining both intra-modality part-level and cross-modality graph-level contextual cues for VI-ReID. We propose an intra-modality weighted-part attention module to extract discriminative part-aggregated features, by imposing the domain knowledge on the part relationship mining. To enhance robustness against noisy samples, we introduce cross-modality graph structured attention to reinforce the representation with the contextual relations across the two modalities. We also develop a parameter-free dynamic dual aggregation learning strategy to adaptively integrate the two components in a progressive joint training manner. Extensive experiments demonstrate that DDAG outperforms the state-of-the-art methods under various settings.

Motivation & Objective

• Motivate and address the challenges of VI-ReID, including large intra-class variations, cross-modality gaps, and substantial noise in data.
• Develop discriminative part-level features within each modality to improve robustness to background clutter.
• Leverage cross-modality graph relations to reinforce representations across visible and infrared images.
• Propose a parameter-free dynamic training strategy to jointly optimize the two attentive components.

Proposed method

• Two-stream backbone with modality-specific first block and shared deeper blocks to learn modality-sharable mid-level features.
• Intra-modality Weighted-Part Aggregation (IWPA) that learns part-level attention within each modality using a non-local style mechanism on p body parts and fuses them with a residual BatchNorm (RBN) weighted aggregation.
• Cross-modality Graph Structured Attention (CGSA) that builds a cross-modality graph on a batch of 2mn images, applies multi-head graph attention to capture cross-modal neighborhood relations, and outputs graph-attention-enhanced features.
• Dynamic dual aggregation learning that treats instance-level part-aggregated learning as dominant (L_P) and progressively adds graph-level cross-modality learning (L_g) via a parameter-free schedule (L^t = L_P^t + 1/(1+E[L_P^{t-1}]) * L_g^t).
• Key equations include: intra-modality part attention map alpha^p_{i,j} = f(x^p_i, x^p_j) / sum_j f(x^p_i, x^p_j) with f = exp(u(x^p_i)^T v(x^p_j)); residual BN aggregation x^* = BN(x^o) + sum_i w^p_i x̄^p_i; graph attention alpha^g_{i,j} computed from transformed features h(x^o_i), h(x^o_j) with multi-head heads; final graph-feature x^g_i = φ(concat_heads sum_j alpha^g_{i,j} h^l(x^o_j)).

Experimental results

Research questions

• RQ1 Can intra-modality part-level attention improve discriminability and robustness to clutter in VI-ReID?
• RQ2 Does incorporating cross-modality graph-structured attention enhance跨-modality feature learning and reduce modality gap?
• RQ3 Does a dynamic, parameter-free aggregation strategy effectively integrate both attentions during training without instability?
• RQ4 How does the proposed DDAG framework compare to state-of-the-art VI-ReID methods on SYSU-MM01 and RegDB datasets?

Key findings

• DDAG outperforms state-of-the-art VI-ReID methods on SYSU-MM01 and RegDB under multiple settings.
• IWPA with residual BN and learnable part weights improves rank-1, rank-5, rank-10, rank-20, and mAP over the baseline.
• CGSA further boosts performance by leveraging cross-modality neighborhood relations and stabilizing training.
• Dynamic dual aggregation without extra hyperparameters effectively combines instance-level and graph-level learning, yielding additive gains.