[Paper Review] Joint Disentangling and Adaptation for Cross-Domain Person Re-Identification

Yang Zou, Xiaodong Yang+3 more|arXiv (Cornell University)|Jul 20, 2020

Video Surveillance and Tracking MethodsComputer Science41 references19 citations

TL;DR

This paper proposes a joint disentangling and adaptation framework for cross-domain person re-identification that separates identity-related (appearance) and identity-unrelated (pose, background, lighting) features, enabling targeted adaptation solely on the identity-relevant space. By co-designing a cycle-consistent image generation module with adversarial alignment and self-training on the disentangled appearance space, the method achieves state-of-the-art performance with significant gains over existing unsupervised domain adaptation approaches.

ABSTRACT

Although a significant progress has been witnessed in supervised person re-identification (re-id), it remains challenging to generalize re-id models to new domains due to the huge domain gaps. Recently, there has been a growing interest in using unsupervised domain adaptation to address this scalability issue. Existing methods typically conduct adaptation on the representation space that contains both id-related and id-unrelated factors, thus inevitably undermining the adaptation efficacy of id-related features. In this paper, we seek to improve adaptation by purifying the representation space to be adapted. To this end, we propose a joint learning framework that disentangles id-related/unrelated features and enforces adaptation to work on the id-related feature space exclusively. Our model involves a disentangling module that encodes cross-domain images into a shared appearance space and two separate structure spaces, and an adaptation module that performs adversarial alignment and self-training on the shared appearance space. The two modules are co-designed to be mutually beneficial. Extensive experiments demonstrate that the proposed joint learning framework outperforms the state-of-the-art methods by clear margins.

Motivation & Objective

• To address the challenge of domain shift in person re-identification, where models trained on one domain fail on new domains due to large domain gaps.
• To overcome the limitation of existing unsupervised domain adaptation methods that operate on mixed feature spaces, where id-unrelated factors interfere with id-related feature learning.
• To jointly optimize feature disentanglement and domain adaptation so that each module enhances the other.
• To enable effective adaptation by focusing exclusively on the disentangled identity-related feature space, minimizing interference from domain-specific variations.

Proposed method

• A cross-domain, cycle-consistent image generation framework with three latent spaces: a shared appearance space (id-related), and separate source and target structure spaces (id-unrelated).
• The disentangling module uses three encoders to decompose images into appearance and structure codes, enabling image reconstruction and cross-domain translation.
• Adaptation is performed exclusively on the shared appearance space via adversarial alignment and self-training using pseudo-labels from the target domain.
• The framework co-trains disentanglement and adaptation modules, where improved disentanglement enhances adaptation, and better adaptation improves disentanglement.
• Loss functions include cycle consistency loss, identity classification loss on synthesized images, and adversarial loss for domain alignment in the appearance space.
• The model is trained end-to-end with hyperparameters tuned on Market1501 → DukeMTMC, showing robustness to variations in λ_cyc and λ_id.

Experimental results

Research questions

• RQ1 Can disentangling identity-related and identity-unrelated features improve unsupervised domain adaptation in cross-domain person re-identification?
• RQ2 Does focusing adaptation solely on the disentangled identity-related feature space lead to better performance than joint adaptation on mixed feature spaces?
• RQ3 Can joint optimization of disentanglement and adaptation create a mutually beneficial training process?
• RQ4 How effective is the proposed method in handling large domain gaps across diverse benchmarks?

Key findings

• The proposed method achieves state-of-the-art performance on six cross-domain person re-identification benchmarks, outperforming existing unsupervised domain adaptation methods by clear margins.
• Abllation studies show that removing cross-domain disentangling or pseudo-identity supervision leads to poor image synthesis quality, confirming the necessity of the joint framework.
• Qualitative results demonstrate that the synthesized images preserve accurate clothing color and style, while realistically transferring background and lighting, unlike baseline methods that fail on foreground or structure transfer.
• The model is robust to hyperparameter changes, with performance stable across λ_cyc ∈ [1,4] and λ_id ∈ [0.25,1], justifying the choice of λ_cyc=2 and λ_id=0.5.
• The method generalizes well across diverse domain pairs, including those with significant differences in clothing style, season, viewpoint, and illumination.