[Paper Review] Unsupervised Person Re-identification by Soft Multilabel Learning

Hong-Xing Yu, Wei‐Shi Zheng+4 more|arXiv (Cornell University)|Mar 15, 2019

Video Surveillance and Tracking MethodsComputer Science59 references61 citations

TL;DR

The paper introduces MAR, a deep model that learns soft multilabel representations for unlabeled target data using auxiliary reference persons, enabling soft multilabel-guided hard negative mining and cross-view consistency to achieve state-of-the-art unsupervised RE-ID on Market-1501 and DukeMTMC-reID.

ABSTRACT

Although unsupervised person re-identification (RE-ID) has drawn increasing research attentions due to its potential to address the scalability problem of supervised RE-ID models, it is very challenging to learn discriminative information in the absence of pairwise labels across disjoint camera views. To overcome this problem, we propose a deep model for the soft multilabel learning for unsupervised RE-ID. The idea is to learn a soft multilabel (real-valued label likelihood vector) for each unlabeled person by comparing (and representing) the unlabeled person with a set of known reference persons from an auxiliary domain. We propose the soft multilabel-guided hard negative mining to learn a discriminative embedding for the unlabeled target domain by exploring the similarity consistency of the visual features and the soft multilabels of unlabeled target pairs. Since most target pairs are cross-view pairs, we develop the cross-view consistent soft multilabel learning to achieve the learning goal that the soft multilabels are consistently good across different camera views. To enable effecient soft multilabel learning, we introduce the reference agent learning to represent each reference person by a reference agent in a joint embedding. We evaluate our unified deep model on Market-1501 and DukeMTMC-reID. Our model outperforms the state-of-the-art unsupervised RE-ID methods by clear margins. Code is available at https://github.com/KovenYu/MAR.

Motivation & Objective

• Address scalability in RE-ID by removing target labels and leveraging an auxiliary labeled dataset.
• Learn soft, real-valued multilabel representations for unlabeled target images via reference agents.
• Develop a unified deep model (MAR) that combines soft multilabel learning, soft-label guided mining, and cross-domain alignment.
• Ensure cross-view consistency of soft multilabels across camera views to improve cross-view matching.

Proposed method

• Define soft multilabel y as a normalized vector of reference-person likelihoods using l(f(x), {a_i}) with y^(k) = exp(a_k^T f(x)) / sum_i exp(a_i^T f(x)).
• Perform soft multilabel-guided hard negative mining by classifying pairs into P/N based on feature similarity and soft multilabel agreement A(y_i, y_j) = sum_k min(y_i^(k), y_j^(k)).
• Introduce cross-view consistent soft multilabel learning loss L_CML to align soft multilabel distributions across camera views via a 2-Wasserstein distance simplification.
• Use reference agents {a_i} to compactly represent reference persons and define the Agent Learning loss L_AL to align auxiliary data; include joint embedding via L_RAJ to rectify cross-domain misalignment.
• Train MAR with L_MDL + λ1 L_CML + λ2 L_RAL where L_MDL enforces discriminative embedding through soft multilabel guided mining.
• Evaluate by computing cosine similarities for probe-gallery pairs at test time.

Experimental results

Research questions

• RQ1 How can soft, real-valued multilabels learned from an auxiliary reference set improve unsupervised RE-ID in a target domain?
• RQ2 Can soft multilabel-guided hard negative mining and cross-view consistency enhance discriminative embedding learning without target labels?
• RQ3 Do reference agents enable efficient and robust cross-domain learning between auxiliary and target datasets?
• RQ4 What performance gains are achievable on standard unsupervised RE-ID benchmarks (Market-1501, DukeMTMC-reID) using MAR?

Key findings

• MAR achieves state-of-the-art unsupervised RE-ID performance on Market-1501 and DukeMTMC-reID, outperforming prior methods by notable margins.
• On Market-1501, MAR achieves Rank-1 67.7, Rank-5 81.9, and mAP 40.0.
• On DukeMTMC-reID, MAR achieves Rank-1 67.1, Rank-5 79.8, and mAP 48.0.
• Ablation shows cross-view soft multilabel learning and reference agent learning are essential for performance gains beyond a pretrained source-only model.
• Soft multilabel guidance substantially improves over feature-similarity-based baselines, and removing L_CML or L_RAL degrades results.
• Visual analyses demonstrate MAR identifies fine-grained discriminative cues (e.g., backpack presence) to distinguish visually similar individuals.