[Paper Review] Person Transfer GAN to Bridge Domain Gap for Person Re-Identification

Longhui Wei, Shiliang Zhang+2 more|arXiv (Cornell University)|Nov 23, 2017

Video Surveillance and Tracking Methods29 references118 citations

TL;DR

Introduces MSMT17, a large-scale multi-scene/multi-time dataset for person ReID, and PTGAN, a GAN-based method to transfer persons across datasets to bridge domain gaps while preserving identity.

ABSTRACT

Although the performance of person Re-Identification (ReID) has been significantly boosted, many challenging issues in real scenarios have not been fully investigated, e.g., the complex scenes and lighting variations, viewpoint and pose changes, and the large number of identities in a camera network. To facilitate the research towards conquering those issues, this paper contributes a new dataset called MSMT17 with many important features, e.g., 1) the raw videos are taken by an 15-camera network deployed in both indoor and outdoor scenes, 2) the videos cover a long period of time and present complex lighting variations, and 3) it contains currently the largest number of annotated identities, i.e., 4,101 identities and 126,441 bounding boxes. We also observe that, domain gap commonly exists between datasets, which essentially causes severe performance drop when training and testing on different datasets. This results in that available training data cannot be effectively leveraged for new testing domains. To relieve the expensive costs of annotating new training samples, we propose a Person Transfer Generative Adversarial Network (PTGAN) to bridge the domain gap. Comprehensive experiments show that the domain gap could be substantially narrowed-down by the PTGAN.

Motivation & Objective

• Create a challenging, realistic large-scale dataset (MSMT17) for person ReID in varied scenes, times, and lighting.
• Address domain gap between ReID datasets to improve cross-domain learning without extensive labeling.
• Propose PTGAN to transfer persons between datasets while preserving identity and adopting style transfer.
• Evaluate PTGAN’s effectiveness in reducing domain gap and enabling cross-dataset training strategies.

Proposed method

• Propose PTGAN, an unpaired image-to-image translation framework based on Cycle-GAN with added identity constraints.
• Use style loss (GAN + cycle-consistency) to map styles between datasets A and B and enforce cycle consistency.
• Incorporate an identity loss that preserves foreground person identity during transfer using a foreground mask M and PSPNet-based segmentation.
• Train using adversarial losses with discriminators for both domains and an L2 foreground fidelity term; set lambda parameters to balance style and identity.
• Demonstrate that PTGAN does not require labeled data in the target dataset and can produce high-quality transferred samples maintaining identities.

Experimental results

Research questions

• RQ1 Can a generative transfer model reduce domain gaps between person ReID datasets by re-styling foregrounds to resemble a target domain?
• RQ2 Does preserving person identity during transfer enable effective cross-dataset training for ReID without labeling the target domain?
• RQ3 How much can cross-dataset training performance improve when using PTGAN-generated samples compared to using only source-domain data?

Key findings

• MSMT17 is a large, challenging ReID dataset with 4,101 identities and 126,441 bounding boxes across 15 cameras.
• PTGAN substantially narrows cross-dataset domain gaps and improves cross-domain ReID performance.
• Replacing or augmenting training data with PTGAN-transferred samples yields significant Rank-1 gains on target datasets without target labels.
• On MSMT17, transferring from Duke, Market, and CUHK03 to MSMT17 improves Rank-1 by 3.7–6.8 percentage points (example: from Duke to MSMT17).
• Combining transferred samples from multiple sources for training provides additional gains over single-source transfer.
• Transfers can make transferred data roughly equivalent to a fraction of MSMT17's own annotated data, reducing labeling costs.