[Paper Review] Interaction-and-Aggregation Network for Person Re-identification

Ruibing Hou, Bingpeng Ma+4 more|arXiv (Cornell University)|Jul 19, 2019

Video Surveillance and Tracking Methods51 references39 citations

TL;DR

The paper introduces Interaction-and-Aggregation (IA) blocks comprising Spatial IA (SIA) and Channel IA (CIA) to adaptively model spatial and channel dependencies, enhancing CNNs for person re-identification and achieving state-of-the-art results on multiple benchmarks.

ABSTRACT

Person re-identification (reID) benefits greatly from deep convolutional neural networks (CNNs) which learn robust feature embeddings. However, CNNs are inherently limited in modeling the large variations in person pose and scale due to their fixed geometric structures. In this paper, we propose a novel network structure, Interaction-and-Aggregation (IA), to enhance the feature representation capability of CNNs. Firstly, Spatial IA (SIA) module is introduced. It models the interdependencies between spatial features and then aggregates the correlated features corresponding to the same body parts. Unlike CNNs which extract features from fixed rectangle regions, SIA can adaptively determine the receptive fields according to the input person pose and scale. Secondly, we introduce Channel IA (CIA) module which selectively aggregates channel features to enhance the feature representation, especially for smallscale visual cues. Further, IA network can be constructed by inserting IA blocks into CNNs at any depth. We validate the effectiveness of our model for person reID by demonstrating its superiority over state-of-the-art methods on three benchmark datasets.

Motivation & Objective

• Address pose and scale variations that challenge fixed CNN receptive fields in person reID.
• Propose SIA to adaptively localize body parts by learning spatial semantic relations.
• Propose CIA to aggregate channel-wise features for small-scale cues.
• Integrate IA blocks into CNN backbones to form the IA Network (IANet).
• Demonstrate superior performance over state-of-the-art methods on standard reID datasets.

Proposed method

• Define Spatial IA (SIA) to compute appearance and location relations and aggregate semantically related spatial features.
• Define Channel IA (CIA) to compute channel-wise semantic relations and aggregate semantically similar channel features.
• Combine SIA and CIA into IA blocks with a residual formulation that can be inserted at network bottlenecks.
• Insert IA blocks into ResNet-50 to build IANet and train end-to-end with cross-entropy loss for identity classification.
• Evaluate on CUHK03, Market-1501, DukeMTMC-reID, and MSMT17 using mean Average Precision (mAP) and CMC top-k metrics.

Experimental results

Research questions

• RQ1 Can adaptive spatial receptive fields via SIA improve body-part localization under pose/scale variation without external part detectors?
• RQ2 Does modeling channel interdependencies via CIA enhance discrimination of small-scale cues (e.g., bags, shoes) in reID?
• RQ3 Do IA blocks placed at network bottlenecks yield better gains than internal block placements across multiple backbones?

Key findings

• IANet outperforms state-of-the-art on Market-1501 (top-1: 94.4, mAP: 83.1) and DukeMTMC (top-1: 87.1, mAP: 73.4).
• On MSMT17, IANet achieves top-1 75.5, top-5 85.5, top-10 88.7, and mAP 46.8, surpassing prior methods.
• Ablation shows multi-context SIA improves performance over single-context, and combining SIA with CIA yields the best results.
• Placing IA blocks at stage-2 and stage-3 bottlenecks provides strong gains with modest parameter overhead.
• IA blocks provide robustness to imperfect person detection and outperform attention-based and multi-scale baselines.