[Paper Review] Deep Transfer Learning for Person Re-identification

Mengyue Geng, Yaowei Wang+2 more|arXiv (Cornell University)|Nov 16, 2016

Video Surveillance and Tracking Methods40 references218 citations

TL;DR

This paper presents a deep transfer learning framework for person Re-ID that uses an ImageNet-pretrained two-loss Siamese network, a two-step fine-tuning strategy, and a co-training based unsupervised transfer, achieving state-of-the-art Rank-1 results on several datasets.

ABSTRACT

Person re-identification (Re-ID) poses a unique challenge to deep learning: how to learn a deep model with millions of parameters on a small training set of few or no labels. In this paper, a number of deep transfer learning models are proposed to address the data sparsity problem. First, a deep network architecture is designed which differs from existing deep Re-ID models in that (a) it is more suitable for transferring representations learned from large image classification datasets, and (b) classification loss and verification loss are combined, each of which adopts a different dropout strategy. Second, a two-stepped fine-tuning strategy is developed to transfer knowledge from auxiliary datasets. Third, given an unlabelled Re-ID dataset, a novel unsupervised deep transfer learning model is developed based on co-training. The proposed models outperform the state-of-the-art deep Re-ID models by large margins: we achieve Rank-1 accuracy of 85.4\%, 83.7\% and 56.3\% on CUHK03, Market1501, and VIPeR respectively, whilst on VIPeR, our unsupervised model (45.1\%) beats most supervised models.

Motivation & Objective

• Motivate Re-ID as a data-sparse deep learning problem due to small labelled datasets.
• Design a network architecture that transfers representations learned on ImageNet to Re-ID tasks.
• Develop training strategies (two-step fine-tuning) to bridge task and domain gaps.
• Introduce an unsupervised, co-training based transfer learning model for unlabelled Re-ID data.

Proposed method

• Use a two-branch Siamese network with a shared base network (GoogLeNet).
• Combine an identity classification loss with a pairwise verification loss to learn transferable features.
• Apply loss-specific dropout: standard dropout for classification and pairwise-consistent dropout for verification.
• Propose a two-stepped fine-tuning strategy (first fine-tune the new classification layer, then fine-tune all layers).
• Develop a co-training based unsupervised domain adaptation framework combining a graph-regularised subspace learning model with self-training deep CNNs.

Experimental results

Research questions

• RQ1 Can ImageNet-derived representations be effectively transferred to person Re-ID with a dual-loss objective?
• RQ2 Does a two-step fine-tuning strategy improve transfer from ImageNet to large and then small Re-ID datasets?
• RQ3 Can unsupervised domain adaptation via co-training outperform supervised methods on Re-ID benchmarks?

Key findings

• The proposed model achieves Rank-1 accuracies of 85.4%, 83.7%, and 56.3% on CUHK03, Market1501, and VIPeR respectively.
• On VIPeR, the unsupervised model (45.1%) beats most supervised models, showing strong unsupervised transfer performance.
• The two-loss Siamese architecture with two different dropout strategies improves Re-ID accuracy by about 3% with pairwise-consistent dropout.
• The two-stepped fine-tuning strategy outperforms conventional one-stepped fine-tuning, especially when target data is small.
• Across 5 benchmarks, the proposed models outperform state-of-the-art deep Re-ID models by a large margin.