[Paper Review] Dual Gaussian-based Variational Subspace Disentanglement for Visible-Infrared Person Re-Identification

Nan Pu, Wei Chen+3 more|arXiv (Cornell University)|Aug 6, 2020

Video Surveillance and Tracking Methods52 references6 citations

TL;DR

This paper proposes a dual Gaussian-based variational auto-encoder (DG-VAE) that disentangles identity-discriminable (IDI) and identity-ambiguous (IAI) features for visible-infrared person re-identification (VI-ReID). By modeling IDI with a mixture-of-Gaussians prior and IAI with a standard Gaussian prior, and using a triplet swap reconstruction strategy, the method achieves state-of-the-art performance on two benchmark datasets, improving mAP by 13.05% on SYSU-MM01 and 10.86% on RegDB.

ABSTRACT

Visible-infrared person re-identification (VI-ReID) is a challenging and essential task in night-time intelligent surveillance systems. Except for the intra-modality variance that RGB-RGB person re-identification mainly overcomes, VI-ReID suffers from additional inter-modality variance caused by the inherent heterogeneous gap. To solve the problem, we present a carefully designed dual Gaussian-based variational auto-encoder (DG-VAE), which disentangles an identity-discriminable and an identity-ambiguous cross-modality feature subspace, following a mixture-of-Gaussians (MoG) prior and a standard Gaussian distribution prior, respectively. Disentangling cross-modality identity-discriminable features leads to more robust retrieval for VI-ReID. To achieve efficient optimization like conventional VAE, we theoretically derive two variational inference terms for the MoG prior under the supervised setting, which not only restricts the identity-discriminable subspace so that the model explicitly handles the cross-modality intra-identity variance, but also enables the MoG distribution to avoid posterior collapse. Furthermore, we propose a triplet swap reconstruction (TSR) strategy to promote the above disentangling process. Extensive experiments demonstrate that our method outperforms state-of-the-art methods on two VI-ReID datasets.

Motivation & Objective

• Address the challenge of cross-modality variance in visible-infrared person re-identification (VI-ReID), which arises from inherent spectral differences between visible and infrared modalities.
• Overcome the limitations of standard VAEs in modeling intra-identity variations and inter-class separability in cross-modality settings.
• Disentangle identity-discriminable features (IDI) from identity-ambiguous features (IAI) to improve robustness and retrieval accuracy.
• Develop a theoretically grounded variational inference framework for mixture-of-Gaussians priors under supervised settings to prevent posterior collapse.
• Introduce a triplet swap reconstruction (TSR) strategy to enhance identity-consistency during disentanglement and representation learning.

Proposed method

• Design a dual Gaussian-based VAE (DG-VAE) that uses a mixture-of-Gaussians (MoG) prior for identity-discriminable features (IDI) and a standard Gaussian prior for identity-ambiguous features (IAI).
• The MoG prior models intra-identity variations by assigning each identity to a distinct Gaussian component, enhancing inter-class separability.
• Derive two variational inference terms for the MoG prior under supervised learning to stabilize training and avoid posterior collapse.
• Implement a triplet swap reconstruction (TSR) strategy that reconstructs features from swapped triplets (anchor, positive, negative) to enforce identity-consistent feature learning.
• Reconstruct feature maps rather than raw images to reduce computational cost and improve generalization, with minimal performance loss.
• Use a PatchGAN discriminator to assess reconstructed feature maps, though no binary cross-entropy loss is applied to the discriminator in the final model.

Experimental results

Research questions

• RQ1 Can a dual Gaussian prior structure effectively disentangle identity-discriminable and identity-ambiguous features in cross-modality person re-identification?
• RQ2 How does modeling IDI with a mixture-of-Gaussians prior improve intra-class compactness and inter-class separation compared to standard Gaussian priors?
• RQ3 To what extent does the proposed triplet swap reconstruction (TSR) strategy enhance identity-consistency and performance in VI-ReID?
• RQ4 Why is feature map reconstruction more effective and efficient than image reconstruction in this disentanglement framework?
• RQ5 How does the derived variational inference term for MoG prior prevent posterior collapse and improve disentanglement in supervised VI-ReID?

Key findings

• The DG-VAE model achieves a 13.05% improvement in mAP on the SYSU-MM01 dataset compared to the baseline, demonstrating strong robustness to cross-modality variance.
• On the RegDB dataset, the method improves mAP by 10.86% over the baseline, confirming its effectiveness in handling challenging cross-modality discrepancies.
• The MoG prior significantly enhances feature disentanglement, as shown by a larger inter-class distance margin and more compact intra-class clusters in the latent space.
• The triplet swap reconstruction (TSR) strategy effectively promotes identity-consistent feature learning, leading to improved performance on both datasets.
• Reconstructing feature maps instead of raw images reduces model parameters by half and FLOPs by 80%, while maintaining competitive performance.
• The proposed MoG-based variational inference term successfully avoids posterior collapse and enables stable training with improved disentanglement efficiency.