[Paper Review] Horizontal Pyramid Matching for Person Re-identification

Yang Fu, Yunchao Wei+6 more|arXiv (Cornell University)|Apr 14, 2018

Video Surveillance and Tracking MethodsComputer Science32 references59 citations

TL;DR

Introduces Horizontal Pyramid Matching (HPM) that learns partial, multi-scale, horizontal-part features using both average and max pooling, achieving state-of-the-art re-identification results without post-processing.

ABSTRACT

Despite the remarkable recent progress, person re-identification (Re-ID) approaches are still suffering from the failure cases where the discriminative body parts are missing. To mitigate such cases, we propose a simple yet effective Horizontal Pyramid Matching (HPM) approach to fully exploit various partial information of a given person, so that correct person candidates can be still identified even even some key parts are missing. Within the HPM, we make the following contributions to produce a more robust feature representation for the Re-ID task: 1) we learn to classify using partial feature representations at different horizontal pyramid scales, which successfully enhance the discriminative capabilities of various person parts; 2) we exploit average and max pooling strategies to account for person-specific discriminative information in a global-local manner. To validate the effectiveness of the proposed HPM, extensive experiments are conducted on three popular benchmarks, including Market-1501, DukeMTMC-ReID and CUHK03. In particular, we achieve mAP scores of 83.1%, 74.5% and 59.7% on these benchmarks, which are the new state-of-the-arts. Our code is available on Github

Motivation & Objective

• Motivate robust Re-ID under partial or missing body parts and misalignment.
• Propose a simple, end-to-end framework to exploit partial information via horizontal pyramid scales.
• Combine global average pooling and global max pooling to capture both contextual and discriminative signals.
• Demonstrate state-of-the-art performance on Market-1501, DukeMTMC-ReID, and CUHK03 without post-processing.

Proposed method

• Split feature maps horizontally into multiple pyramid scales (HPP).
• Learn independent classifiers for each horizontal bin at each pyramid scale.
• Use both global average pooling and global max pooling within each partition to obtain G_{i,j} = avgpool(F_{i,j}) + maxpool(F_{i,j}).
• Reduce each G_{i,j} to 256-d via a conv layer to form H_{i,j}, then feed into per-bin classifiers.
• Train with a sum of cross-entropy losses across all pyramid bins and scales.
• During testing, concatenate features from all bins across scales for final representation.

Experimental results

Research questions

• RQ1 Can horizontal multi-scale partitioning improve robustness to misalignment and missing body parts in person Re-ID?
• RQ2 Does combining average and max pooling within each partition enhance discriminative power?
• RQ3 What are the optimal pyramid scales and pooling configurations for competitive end-to-end Re-ID performance?
• RQ4 How does HPM compare to state-of-the-art methods on Market-1501, DukeMTMC-ReID, and CUHK03 without post-processing?

Key findings

• HPM achieves mAP of 83.1% on Market-1501, 74.5% on DukeMTMC-ReID, and 59.7% on CUHK03 with new protocol (state-of-the-art at the time).
• HPM outperforms PCB and PCB+RPP baselines in mAP and Rank-1 on Market-1501 and DukeMTMC-ReID without post-processing.
• Using four pyramid scales with a mix of max and average pooling provides the best trade-off between global and local features.
• Max pooling generally yields better results than average pooling, and combining both pooling strategies yields further gains.
• Ablation studies show four pyramid scales balance global and local information; more scales do not yield consistent improvements and increase cost.