[Paper Review] Learning Disentangled Representation for Robust Person Re-identification

Chanho Eom, Bumsub Ham|arXiv (Cornell University)|Oct 26, 2019

Video Surveillance and Tracking Methods36 citations

TL;DR

IS-GAN learns disentangled identity-related and identity-unrelated features without auxiliary signals, using identity shuffling to achieve state-of-the-art results on standard reID benchmarks.

ABSTRACT

We address the problem of person re-identification (reID), that is, retrieving person images from a large dataset, given a query image of the person of interest. A key challenge is to learn person representations robust to intra-class variations, as different persons can have the same attribute and the same person's appearance looks different with viewpoint changes. Recent reID methods focus on learning discriminative features but robust to only a particular factor of variations (e.g., human pose), which requires corresponding supervisory signals (e.g., pose annotations). To tackle this problem, we propose to disentangle identity-related and -unrelated features from person images. Identity-related features contain information useful for specifying a particular person (e.g., clothing), while identity-unrelated ones hold other factors (e.g., human pose, scale changes). To this end, we introduce a new generative adversarial network, dubbed \emph{identity shuffle GAN} (IS-GAN), that factorizes these features using identification labels without any auxiliary information. We also propose an identity-shuffling technique to regularize the disentangled features. Experimental results demonstrate the effectiveness of IS-GAN, significantly outperforming the state of the art on standard reID benchmarks including the Market-1501, CUHK03 and DukeMTMC-reID. Our code and models are available online: https://cvlab-yonsei.github.io/projects/ISGAN/.

Motivation & Objective

• Motivate robust person re-identification under large intra-class variations and viewpoint changes.
• Disentangle identity-related information from identity-unrelated factors using only identity labels.
• Propose a GAN-based framework (IS-GAN) with identity shuffling to regularize disentanglement.
• Demonstrate improved reID performance on standard benchmarks with ablation studies.

Proposed method

• Propose identity-related and identity-unrelated feature encoders within a GAN framework.
• Disentangle features by learning from pairs of images of the same identity without additional supervision.
• Use an identity shuffling mechanism to generate images by swapping identity-related features across images and across horizontal body-part regions.
• Train with dual discriminators (domain and class) to enforce realistic synthesis and correct identity labeling.
• Optimize a composite loss including identification loss, KL regularization for the unrelated features, and reconstruction-based and adversarial losses (L_S, L_PS, L_D, L_C).

Experimental results

Research questions

• RQ1 Can identity-related and identity-unrelated features be disentangled using only identification labels without auxiliary signals?
• RQ2 Does identity shuffling across whole images and part-level regions improve robustness to pose, occlusion, and background clutter?
• RQ3 How do the disentangled features contribute to improved reID performance on standard benchmarks?
• RQ4 What is the impact of the proposed discriminators and loss components on feature disentanglement and retrieval accuracy?

Key findings

• IS-GAN achieves state-of-the-art rank-1 and mAP on Market-1501 (95.2% R1, 87.1% mAP).
• IS-GAN achieves strong performance on CUHK03 (74.1% R1, 72.5% mAP) and DukeMTMC-reID (72.3% R1, 68.8% mAP) among reported results.
• Ablation studies show that identity disentangling with identity shuffling improves performance across datasets.
• Part-level identity shuffling further boosts reID accuracy compared to full-image shuffling.
• Discriminators for domain and class information complement the disentanglement losses to achieve best overall results.
• IS-GAN demonstrates robustness to pose variations, occlusion, and background clutter through learned identity-related features.