[Paper Review] Identity-Guided Human Semantic Parsing for Person Re-Identification

Kuan Zhu, Haiyun Guo+3 more|arXiv (Cornell University)|Jul 27, 2020

Video Surveillance and Tracking Methods33 references36 citations

TL;DR

ISP uses identity-guided cascaded clustering to learn pixel-level parts including personal belongings, enabling occlusion-robust, alignment-free re-ID without extra semantic supervision.

ABSTRACT

Existing alignment-based methods have to employ the pretrained human parsing models to achieve the pixel-level alignment, and cannot identify the personal belongings (e.g., backpacks and reticule) which are crucial to person re-ID. In this paper, we propose the identity-guided human semantic parsing approach (ISP) to locate both the human body parts and personal belongings at pixel-level for aligned person re-ID only with person identity labels. We design the cascaded clustering on feature maps to generate the pseudo-labels of human parts. Specifically, for the pixels of all images of a person, we first group them to foreground or background and then group the foreground pixels to human parts. The cluster assignments are subsequently used as pseudo-labels of human parts to supervise the part estimation and ISP iteratively learns the feature maps and groups them. Finally, local features of both human body parts and personal belongings are obtained according to the selflearned part estimation, and only features of visible parts are utilized for the retrieval. Extensive experiments on three widely used datasets validate the superiority of ISP over lots of state-of-the-art methods. Our code is available at https://github.com/CASIA-IVA-Lab/ISP-reID.

Motivation & Objective

• Motivate and address misalignment in person re-ID without relying on pre-trained parsing models.
• Introduce a weakly-supervised, identity-guided method to learn pixel-level body parts and personal belongings.
• Enable occlusion-robust alignment by identifying visible parts and excluding occluded regions during matching.
• Demonstrate state-of-the-art performance on multiple holistic and occluded re-ID benchmarks.

Proposed method

• Develop cascaded clustering on backbone feature maps to generate pseudo-labels for human parts including belongings.
• Compute pixel-wise activations from global feature maps to separate foreground/background and cluster foreground pixels across all images of the same identity.
• Iteratively refine part representations via pseudo-labels and part-aware pooling to obtain localized features for body parts and belongings.
• Construct part-level matching using only shared-visible parts in probe-gallery pairs, augmented with global and foreground features.
• Train with a combination of triplet loss and cross-entropy with label smoothing for part, foreground, and global representations, plus a weakly-supervised parsing loss.

Experimental results

Research questions

• RQ1 Can identity-guided, pixel-level part parsing locate both body parts and personal belongings without extra semantic supervision?
• RQ2 Does occlusion-aware, part-level matching improve re-ID performance compared with region- or attention-based methods?
• RQ3 How does the number of clustering centers (K) affect re-ID performance and parsing quality?
• RQ4 Is the learned semantic parsing robust to occlusion and misalignment across datasets?

Key findings

• ISP achieves state-of-the-art results on DukeMTMC-reID with Rank-1 89.6, Rank-5 95.5, and mAP 80.0.
• On Market-1501, ISP attains Rank-1 95.3, Rank-5 98.6, and mAP 88.6.
• On CUHK03-NP, ISP achieves Rank-1 76.5 and mAP 74.1.
• In Occluded-DukeMTMC, ISP reaches Rank-1 62.8, Rank-5 78.1, Rank-10 82.9, and mAP 52.3, setting a new state-of-the-art.
• Ablation shows robustness to K in {4,5,6,7,8} with peak performance around K=7, and learned semantic parsing outperforms extra semantic approximations across datasets.
• ISP demonstrates explicit identification of visible parts and excludes occluded regions during matching, yielding meaningful improvements over baseline and several prior methods.