[Paper Review] Deep Learning for Person Re-identification: A Survey and Outlook

Mang Ye, Jianbing Shen+4 more|arXiv (Cornell University)|Jan 13, 2020

Video Surveillance and Tracking Methods120 citations

TL;DR

This survey analyzes closed-world and open-world person Re-ID, proposes the AGW baseline and a new mINP metric, and outlines open challenges and future directions.

ABSTRACT

Person re-identification (Re-ID) aims at retrieving a person of interest across multiple non-overlapping cameras. With the advancement of deep neural networks and increasing demand of intelligent video surveillance, it has gained significantly increased interest in the computer vision community. By dissecting the involved components in developing a person Re-ID system, we categorize it into the closed-world and open-world settings. The widely studied closed-world setting is usually applied under various research-oriented assumptions, and has achieved inspiring success using deep learning techniques on a number of datasets. We first conduct a comprehensive overview with in-depth analysis for closed-world person Re-ID from three different perspectives, including deep feature representation learning, deep metric learning and ranking optimization. With the performance saturation under closed-world setting, the research focus for person Re-ID has recently shifted to the open-world setting, facing more challenging issues. This setting is closer to practical applications under specific scenarios. We summarize the open-world Re-ID in terms of five different aspects. By analyzing the advantages of existing methods, we design a powerful AGW baseline, achieving state-of-the-art or at least comparable performance on twelve datasets for FOUR different Re-ID tasks. Meanwhile, we introduce a new evaluation metric (mINP) for person Re-ID, indicating the cost for finding all the correct matches, which provides an additional criteria to evaluate the Re-ID system for real applications. Finally, some important yet under-investigated open issues are discussed.

Motivation & Objective

• Summarize the major components and challenges of closed-world person Re-ID under deep learning.
• Differentiate closed-world and open-world Re-ID and analyze their respective methodologies.
• Propose a strong baseline (AGW) and a new evaluation metric (mINP) to drive future research.
• Discuss open issues and practical considerations for real-world Re-ID systems.

Proposed method

• Review deep feature representation learning, deep metric learning, and ranking optimization in closed-world Re-ID.
• Introduce and analyze four feature learning strategies: Global, Local, Auxiliary, and Video features.
• Discuss loss functions (identity, verification, triplet, OIM) and sampling strategies for training deep Re-ID models.
• Propose AGW (Attention Generalized mean pooling with Weighted triplet loss) as a powerful baseline.
• Introduce mINP (mean Inverse Negative Penalty) as a supplementary evaluation metric.

Experimental results

Research questions

• RQ1 What are the strengths and limitations of current deep learning approaches for closed-world person Re-ID?
• RQ2 How do open-world settings introduce additional challenges, and which methods address them?
• RQ3 What constitutes a strong, generalizable baseline for Re-ID across datasets and tasks?
• RQ4 How does the new mINP metric complement existing CMC/mAP in evaluating Re-ID systems?

Key findings

• AGW baseline achieves state-of-the-art or competitive performance on twelve datasets across four Re-ID tasks.
• mINP provides an additional criterion by measuring the cost to find all correct matches.
• Open-world challenges include heterogeneous data, end-to-end search, limited/noisy labels, label noise, and open-set retrieval.
• Open issues are identified to bridge the gap between closed-world benchmarks and real-world deployments.