[Paper Review] Harmonious Attention Network for Person Re-Identification

Wei Li, Xiatian Zhu+1 more|arXiv (Cornell University)|Feb 22, 2018

Video Surveillance and Tracking Methods25 references188 citations

TL;DR

HA-CNN jointly learns soft pixel attention and hard regional attention within a lightweight CNN to improve person re-identification under misaligned bounding boxes, outperforming state-of-the-art on three large benchmarks.

ABSTRACT

Existing person re-identification (re-id) methods either assume the availability of well-aligned person bounding box images as model input or rely on constrained attention selection mechanisms to calibrate misaligned images. They are therefore sub-optimal for re-id matching in arbitrarily aligned person images potentially with large human pose variations and unconstrained auto-detection errors. In this work, we show the advantages of jointly learning attention selection and feature representation in a Convolutional Neural Network (CNN) by maximising the complementary information of different levels of visual attention subject to re-id discriminative learning constraints. Specifically, we formulate a novel Harmonious Attention CNN (HA-CNN) model for joint learning of soft pixel attention and hard regional attention along with simultaneous optimisation of feature representations, dedicated to optimise person re-id in uncontrolled (misaligned) images. Extensive comparative evaluations validate the superiority of this new HA-CNN model for person re-id over a wide variety of state-of-the-art methods on three large-scale benchmarks including CUHK03, Market-1501, and DukeMTMC-ReID.

Motivation & Objective

• Motivate robust person re-id under unconstrained misalignment and background clutter from auto-detected bounding boxes.
• Propose a lightweight CNN that jointly learns multi-level attention (soft pixel, soft channel, and hard region) with re-id discriminative learning.
• Introduce cross-attention interaction to maximize complementary information between attention modules and feature representations.
• Demonstrate that joint soft/hard attention yields superior re-id performance with a compact model.

Proposed method

• Introduce Harmonious Attention (HA) module that combines soft spatial, soft channel, and hard regional attention.
• Use a multi-branch HA-CNN with a global branch and multiple local streams sharing early layers to reduce parameters.
• Soft spatial and channel attention are factorised into A^l = S^l × C^l, where S^l is a spatial attention map and C^l a channel attention map, computed via lightweight sub-networks.
• Hard regional attention locates latent discriminative regions via a small transformation matrix; regions are fed into local streams.
• Intra-level and cross-level architecture: hard and soft attention learned at each level, with cross-attention interaction learning (CAIL) enriching local/global features across branches.
• Cross-attention interaction adds global features to local features: X̃_L^{(l,k)} = X_L^{(l,k)} + X_G^{(l,k)} to enhance discrimination under re-id constraints.
• Joint training uses identity classification loss for both global and local branches, enabling end-to-end learning without large data augmentation or pre-training.

Experimental results

Research questions

• RQ1 How can multi-level attention (soft pixel, soft channel, hard region) be learned jointly within a single re-id model to improve performance under misalignment?
• RQ2 Does cross-attention interaction between global and local feature branches enhance the harmony between attention mechanisms and feature representations?
• RQ3 Can a lightweight HA-CNN achieve state-of-the-art re-id performance with reduced model size and training complexity?

Key findings

• HA-CNN achieves superior Rank-1 and mAP on Market-1501, DukeMTMC-ReID, and CUHK03 compared to a wide range of state-of-the-art methods.
• On Market-1501 (Single-Query), HA-CNN achieves 91.2% R1 and 75.7% mAP, and 93.8% R1 and 82.8% mAP for Multi-Query.
• On DukeMTMC-ReID, HA-CNN achieves 80.5% R1 and 63.8% mAP.
• On CUHK03 (Deteced, 767/700 split), HA-CNN achieves 41.7% mAP and 41.7%? R1?; the reported best among compared methods is 44.4% R1 and 41.0% mAP for labelled and 41.7% R1 and 38.6% mAP for detected (note: numbers extracted as reported).