[Paper Review] Learning to Adapt Invariance in Memory for Person Re-identification

Zhun Zhong, Liang Zheng+3 more|arXiv (Cornell University)|Aug 1, 2019

Video Surveillance and Tracking Methods61 references36 citations

TL;DR

It proposes an unsupervised domain adaptation framework for person re-ID that enforces exemplar-, camera-, and neighborhood-invariance via an exemplar memory and Graph-based Positive Prediction (GPP) to improve target-domain generalization.

ABSTRACT

This work considers the problem of unsupervised domain adaptation in person re-identification (re-ID), which aims to transfer knowledge from the source domain to the target domain. Existing methods are primary to reduce the inter-domain shift between the domains, which however usually overlook the relations among target samples. This paper investigates into the intra-domain variations of the target domain and proposes a novel adaptation framework w.r.t. three types of underlying invariance, i.e., Exemplar-Invariance, Camera-Invariance, and Neighborhood-Invariance. Specifically, an exemplar memory is introduced to store features of samples, which can effectively and efficiently enforce the invariance constraints over the global dataset. We further present the Graph-based Positive Prediction (GPP) method to explore reliable neighbors for the target domain, which is built upon the memory and is trained on the source samples. Experiments demonstrate that 1) the three invariance properties are indispensable for effective domain adaptation, 2) the memory plays a key role in implementing invariance learning and improves the performance with limited extra computation cost, 3) GPP could facilitate the invariance learning and thus significantly improves the results, and 4) our approach produces new state-of-the-art adaptation accuracy on three re-ID large-scale benchmarks.

Motivation & Objective

• Investigate how intra-domain variations in the target domain affect unsupervised domain adaptation for person re-ID.
• Introduce three target invariance properties—Exemplar-Invariance, Camera-Invariance, and Neighborhood-Invariance—and evaluate their impact.
• Develop a memory-based framework to enforce these invariances across the full training set with minimal extra computation.
• Incorporate a Graph-based Positive Prediction (GPP) module to reliably identify positive neighbors for invariant learning.

Proposed method

• Use a dual-branch training framework with a shared feature extractor and domain-specific exemplar memories (source and target).
• Impose three invariance losses on the target branch: exemplar-invariance (each target exemplar classified to its own class), camera-invariance (target image and CamStyle variants share a class), and neighborhood-invariance (target image classified toward reliable neighbors).
• Introduce an exemplar memory that stores up-to-date target features and enables global, non-batch-based invariant learning.
• Propose Graph-based Positive Prediction (GPP) to refine neighbor relations by constructing a graph from top-k memory neighbors and applying Graph Convolutional Networks to predict positive neighbors (trained on source data, applied to target).
• Train GPP on the source domain to learn positive neighbor prediction, then apply to target domain to assist NI learning without backpropagating through the feature extractor.
• Utilize CamStyle (CycleGAN/StarGAN) to generate camera-style variants for target images to support CI.

Experimental results

Research questions

• RQ1 Can incorporating exemplar-, camera-, and neighborhood-invariance improve unsupervised domain adaptation for person re-ID compared to inter-domain distribution alignment alone?
• RQ2 Does an exemplar memory enable effective global (not mini-batch) invariance learning with minimal extra computation?
• RQ3 Does Graph-based Positive Prediction enhance the reliability of neighborhood-invariance learning for target domain adaptation?
• RQ4 How does the proposed method perform on large-scale re-ID benchmarks (Market-1501, DukeMTMC-reID, MSMT17) compared with state-of-the-art UDA approaches?

Key findings

• Three invariance properties are indispensable for effective domain adaptation in re-ID.
• Memory-based invariance learning significantly improves performance with limited additional computation and memory.
• GPP improves invariance learning by providing more reliable positive neighbors for NI, leading to better accuracy (notably mAP) on target data.
• The approach achieves state-of-the-art adaptation accuracy on three large-scale re-ID benchmarks (Market-1501, DukeMTMC-reID, MSMT17).