[Paper Review] Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)

Yifan Sun, Liang Zheng+3 more|arXiv (Cornell University)|Nov 26, 2017

Video Surveillance and Tracking MethodsComputer Science42 references152 citations

TL;DR

This paper introduces PCB, a strong convolutional baseline for person re-identification using uniform part partitioning, and RPP, a refined part pooling method that relocates outlier features to improve within-part consistency, achieving state-of-the-art results on Market-1501, DukeMTMC-reID, and CUHK03 without pose cues.

ABSTRACT

Employing part-level features for pedestrian image description offers fine-grained information and has been verified as beneficial for person retrieval in very recent literature. A prerequisite of part discovery is that each part should be well located. Instead of using external cues, e.g., pose estimation, to directly locate parts, this paper lays emphasis on the content consistency within each part. Specifically, we target at learning discriminative part-informed features for person retrieval and make two contributions. (i) A network named Part-based Convolutional Baseline (PCB). Given an image input, it outputs a convolutional descriptor consisting of several part-level features. With a uniform partition strategy, PCB achieves competitive results with the state-of-the-art methods, proving itself as a strong convolutional baseline for person retrieval. (ii) A refined part pooling (RPP) method. Uniform partition inevitably incurs outliers in each part, which are in fact more similar to other parts. RPP re-assigns these outliers to the parts they are closest to, resulting in refined parts with enhanced within-part consistency. Experiment confirms that RPP allows PCB to gain another round of performance boost. For instance, on the Market-1501 dataset, we achieve (77.4+4.2)% mAP and (92.3+1.5)% rank-1 accuracy, surpassing the state of the art by a large margin.

Motivation & Objective

• Motivate learning discriminative part-informed features for person retrieval without external pose cues.
• Propose PCB to extract part-level features via uniform partitioning of conv-layer outputs.
• Introduce Refined Part Pooling (RPP) to relocate outliers and strengthen within-part consistency.
• Demonstrate that PCB + RPP achieves new state-of-the-art results on major re-ID benchmarks.

Proposed method

• PCB replaces the global pooling with a uniform horizontal partition of the conv feature map, followed by per-part classifiers and a final concatenation of part descriptors.
• PCB uses a backbone (e.g., ResNet-50) with the last spatial down-sampling preserved to increase part granularity; each stripe is pooled to a vector, reduced in dimension, and classified with its own FC+Softmax branch.
• RPP introduces a part classifier to assign each local feature to one of p parts using a Softmax over part scores, then samples features per part to refine the stripe-level partition.
• An induced training procedure initializes with uniform partition, appends a part classifier, fixes backbone during a second phase to train the part classifier, and finally fine-tunes the entire network.
• Compared variants show that independent per-part losses and non-shared classifier parameters are beneficial for discriminative part features.

Experimental results

Research questions

• RQ1 Can a strong convolutional baseline with uniform part partitioning achieve competitive performance in person re-ID without pose or region proposals?
• RQ2 Does refining the uniform partition via a learned part classifier (RPP) improve within-part consistency and overall retrieval metrics?
• RQ3 How does PCB+RPP compare to attention-based or pose-guided partitioning methods on standard re-ID benchmarks?

Key findings

• PCB (uniform partition) yields substantial gains over global-descriptor baselines, setting a strong convolutional baseline for person re-ID.
• RPP further improves performance by relocating outliers to the most similar parts, increasing within-part consistency and boosting mAP.
• PCB+RPP achieves state-of-the-art results on Market-1501 (mAP 81.6, Rank-1 93.1), DukeMTMC-reID (mAP 69.2, Rank-1 83.7), and CUHK03 (mAP 57.5, Rank-1 63.7) without re-ranking.
• Induced training for the part classifier is crucial; without induction, attention-like behavior yields inferior results.
• Sharing FC parameters across part classifiers harms performance; separate per-part classifiers are preferable.
• Compared to IDE baseline, PCB provides notable mAP improvements across datasets (e.g., Market-1501: 68.5→77.4 mAP; Duke: 52.8→66.1 mAP).