[Paper Review] Invariance Matters: Exemplar Memory for Domain Adaptive Person Re-identification

Zhun Zhong, Liang Zheng+3 more|arXiv (Cornell University)|Apr 3, 2019

Video Surveillance and Tracking Methods37 references55 citations

TL;DR

The paper introduces exemplar memory and three target-domain invariances (exemplar-, camera-, neighborhood-invariance) to improve unsupervised domain adaptation for person re-ID, achieving state-of-the-art results on Market-1501, DukeMTMC-reID, and MSMT17.

ABSTRACT

This paper considers the domain adaptive person re-identification (re-ID) problem: learning a re-ID model from a labeled source domain and an unlabeled target domain. Conventional methods are mainly to reduce feature distribution gap between the source and target domains. However, these studies largely neglect the intra-domain variations in the target domain, which contain critical factors influencing the testing performance on the target domain. In this work, we comprehensively investigate into the intra-domain variations of the target domain and propose to generalize the re-ID model w.r.t three types of the underlying invariance, i.e., exemplar-invariance, camera-invariance and neighborhood-invariance. To achieve this goal, an exemplar memory is introduced to store features of the target domain and accommodate the three invariance properties. The memory allows us to enforce the invariance constraints over global training batch without significantly increasing computation cost. Experiment demonstrates that the three invariance properties and the proposed memory are indispensable towards an effective domain adaptation system. Results on three re-ID domains show that our domain adaptation accuracy outperforms the state of the art by a large margin. Code is available at: https://github.com/zhunzhong07/ECN

Motivation & Objective

• Motivate domain-adaptive person re-ID by addressing intra-target-domain variations neglected by prior methods.
• Propose three invariance properties (exemplar-, camera-, neighborhood-invariance) to generalize representations on the target domain.
• Introduce an exemplar memory module to enforce invariance constraints over the global target set with low computation overhead.
• Demonstrate substantial gains over state-of-the-art UDA methods across multiple large-scale datasets.

Proposed method

• Use ResNet-50 backbone with a 4096-dim FC layer (FC-4096) for feature extraction.
• Maintain a supervised classifier on source data with cross-entropy loss.
• Introduce an exemplar memory (key K, value V) storing up-to-date target features; update K during training and normalize.
• Define exemplar-invariance by treating each target image as its own class and maximizing similarity to its own exemplar.
• Enforce camera-invariance by using CamStyle-transferred images to pull same-identity samples across camera styles (via p(i|x̂_t,i)).
• Impose neighborhood-invariance by pulling an exemplar toward its k-nearest neighbors in the memory (soft-label loss on neighbors).
• Combine losses into final objective L = (1-λ)L_src + λL_tgt, with L_tgt aggregating exemplar-, camera-, and neighborhood-invariance losses.

Experimental results

Research questions

• RQ1 How can target-domain intra-class/within-domain variations be leveraged to improve unsupervised domain adaptation for person re-ID?
• RQ2 Do exemplar-, camera-, and neighborhood-invariance collectively improve transferability over prior cross-domain alignment methods?
• RQ3 Can an exemplar memory mechanism efficiently enforce global target-domain invariances during training?
• RQ4 To what extent do the proposed invariances improve performance across diverse re-ID datasets?

Key findings

• Exemplar-invariance, camera-invariance, and neighborhood-invariance collectively improve cross-domain re-ID performance over source-only baselines.
• The exemplar memory enables global target-sample relation modeling with modest extra computation/memory (~260 MB).
• Adding camera-invariance yields large gains (e.g., from 63.1% to 75.1% rank-1 when using Duke as source and Market as target).
• Neighborhood-invariance provides further gains when combined with exemplar- and camera-invariance (e.g., 75.1% rank-1 on Market from Duke→Market with E+C+N).
• ECN achieves state-of-the-art unsupervised domain-adaptation results on Market-1501, DukeMTMC-reID, and MSMT17, outperforming prior methods by notable margins.