[Paper Review] Leveraging Virtual and Real Person for Unsupervised Person Re-identification

Fengxiang Yang, Zhun Zhong+3 more|arXiv (Cornell University)|Nov 5, 2018

Video Surveillance and Tracking Methods12 references4 citations

TL;DR

This paper proposes a novel unsupervised person re-identification framework that leverages synthetic (virtual) and real person images to train deep re-ID models without labels. It generates pseudo-labeled virtual persons using GAN-based pose and style transfer, then iteratively mines reliable positive pairs from real data via collaborative filtering, achieving state-of-the-art performance on Market-1501 and DukeMTMC-reID.

ABSTRACT

Person re-identification (re-ID) is a challenging problem especially when no labels are available for training. Although recent deep re-ID methods have achieved great improvement, it is still difficult to optimize deep re-ID model without annotations in training data. To address this problem, this study introduces a novel approach for unsupervised person re-ID by leveraging virtual and real data. Our approach includes two components: virtual person generation and training of deep re-ID model. For virtual person generation, we learn a person generation model and a camera style transfer model using unlabeled real data to generate virtual persons with different poses and camera styles. The virtual data is formed as labeled training data, enabling subsequently training deep re-ID model in supervision. For training of deep re-ID model, we divide it into three steps: 1) pre-training a coarse re-ID model by using virtual data; 2) collaborative filtering based positive pair mining from the real data; and 3) fine-tuning of the coarse re-ID model by leveraging the mined positive pairs and virtual data. The final re-ID model is achieved by iterating between step 2 and step 3 until convergence. Experimental results on two large-scale datasets, Market-1501 and DukeMTMC-reID, demonstrate the effectiveness of our approach and shows that the state of the art is achieved in unsupervised person re-ID.

Motivation & Objective

• To address the challenge of training deep person re-identification models without labeled training data.
• To improve unsupervised re-ID performance on large-scale datasets where hand-crafted features fail.
• To enable supervised-style training of deep re-ID models using only unlabeled real data and synthetic virtual data.
• To develop a robust method for mining reliable positive image pairs from unlabeled real data.
• To achieve state-of-the-art performance in unsupervised person re-identification on benchmark datasets.

Proposed method

• Generates virtual person images using DPG-GAN for pose variation and Star-GAN for camera style transfer, creating pseudo-labeled training data.
• Pre-trains a coarse re-ID model on the generated virtual data using identity classification loss (L_cls).
• Uses the pre-trained model to extract features and compute k-reciprocal nearest neighbors (k-RNNs) on real data for initial positive pair candidates.
• Applies a collaborative filtering-based approach to filter out false positive pairs and select high-confidence positive pairs from real data.
• Fine-tunes the re-ID model using a multi-task loss combining L_cls on virtual data and triplet loss on mined real positive pairs.
• Iterates between positive pair mining and model fine-tuning until convergence, improving model robustness and accuracy.

Experimental results

Research questions

• RQ1 Can synthetic virtual person images with pseudo-labels effectively pre-train a deep re-ID model in the absence of real annotations?
• RQ2 Can collaborative filtering improve the quality of positive pair mining from unlabeled real data in unsupervised re-ID?
• RQ3 Does combining virtual data with refined real data pairs lead to better generalization and performance on large-scale re-ID benchmarks?
• RQ4 How sensitive is the method to hyperparameters such as virtual data scale, batch size, and loss weighting?
• RQ5 Can the proposed framework achieve state-of-the-art performance in unsupervised person re-identification without any labeled data?

Key findings

• The proposed method achieves state-of-the-art performance on Market-1501 and DukeMTMC-reID under unsupervised settings, outperforming existing methods.
• Collaborative filtering-based mining improves rank-1 accuracy by 6.6% and 2.9% over random selection on Market-1501 and DukeMTMC-reID, respectively.
• The accuracy of mined positive pairs increases from 29.0% and 14.5% (random selection) to 67.5% and 40.5% (collaborative filtering), confirming higher pair quality.
• Model performance saturates when virtual dataset size reaches 36 and 48 identities per person for Market-1501 and DukeMTMC-reID, respectively.
• The optimal loss weighting (λ) for multi-task training is around 1, where both classification and triplet losses contribute equally to performance.
• The method is robust to variations in k and batch size (N_r), with minimal performance fluctuation across tested configurations.