[Paper Review] Calibrated Feature Decomposition for Generalizable Person Re-Identification

Kecheng Zheng, Jiawei Liu+3 more|arXiv (Cornell University)|Nov 27, 2021

Video Surveillance and Tracking Methods42 references4 citations

TL;DR

This paper proposes Calibrated Feature Decomposition (CFD), a novel module for domain generalizable person re-identification that decomposes features into identity-relevant, domain-specific, and entangled components. By introducing calibrated batch normalization (CSBN) and calibrated instance normalization (CIN), CFD enhances feature discrimination and generalization, achieving state-of-the-art performance across multiple benchmarks with significant gains over prior methods.

ABSTRACT

Existing disentangled-based methods for generalizable person re-identification aim at directly disentangling person representations into domain-relevant interference and identity-relevant feature. However, they ignore that some crucial characteristics are stubbornly entwined in both the domain-relevant interference and identity-relevant feature, which are intractable to decompose in an unsupervised manner. In this paper, we propose a simple yet effective Calibrated Feature Decomposition (CFD) module that focuses on improving the generalization capacity for person re-identification through a more judicious feature decomposition and reinforcement strategy. Specifically, a calibrated-and-standardized Batch normalization (CSBN) is designed to learn calibrated person representation by jointly exploring intra-domain calibration and inter-domain standardization of multi-source domain features. CSBN restricts instance-level inconsistency of feature distribution for each domain and captures intrinsic domain-level specific statistics. The calibrated person representation is subtly decomposed into the identity-relevant feature, domain feature, and the remaining entangled one. For enhancing the generalization ability and ensuring high discrimination of the identity-relevant feature, a calibrated instance normalization (CIN) is introduced to enforce discriminative id-relevant information, and filter out id-irrelevant information, and meanwhile the rich complementary clues from the remaining entangled feature are further employed to strengthen it. Extensive experiments demonstrate the strong generalization capability of our framework. Our models empowered by CFD modules significantly outperform the state-of-the-art domain generalization approaches on multiple widely-used benchmarks. Code will be made public: https://github.com/zkcys001/CFD.

Motivation & Objective

• Address the limitation of existing disentanglement-based methods in person re-identification that fail to handle stubbornly entangled features across domains.
• Improve model generalization in open-set domain generalization settings where source and target domains have variable identity counts.
• Develop a more judicious feature decomposition strategy that separates identity-relevant, domain-specific, and entangled features rather than assuming perfect disentanglement.
• Introduce normalization techniques (CSBN and CIN) to guide and reinforce feature decomposition for better domain-invariant representation learning.
• Achieve state-of-the-art performance on multiple benchmarks without relying on domain-specific model ensembles or complex meta-learning mechanisms.

Proposed method

• Propose a Calibrated Feature Decomposition (CFD) module that decomposes features into three components: identity-relevant, domain-specific, and remaining entangled features.
• Design calibrated-and-standardized Batch Normalization (CSBN) to enforce intra-domain consistency and inter-domain standardization, capturing domain-specific statistics while stabilizing feature distributions.
• Introduce Calibrated Instance Normalization (CIN) to enhance discriminative power of identity-relevant features by filtering out id-irrelevant information.
• Use channel-wise attention (CBAM-style) to guide feature decomposition, with ablation showing joint spatial and channel attention (SC) yields the best performance.
• Apply domain loss to guide the decomposition process, improving generalization by aligning features across domains.
• Integrate CFD into ResNeXt-50-MSL backbone across all stages, with ablation showing optimal performance when applied to all stages.

Experimental results

Research questions

• RQ1 Can a more nuanced feature decomposition strategy—separating identity, domain, and entangled components—improve generalization in person re-identification beyond standard disentanglement?
• RQ2 How do calibrated normalization techniques (CSBN and CIN) affect the quality and generalization of decomposed features in unseen domains?
• RQ3 Does the inclusion of domain loss during training improve the effectiveness of feature decomposition in CFD?
• RQ4 What is the impact of different attention mechanisms (spatial, channel, or joint) on the performance of the CFD module?
• RQ5 To what extent does CFD outperform existing domain generalization methods on open-set, variable-identity benchmarks?

Key findings

• The CFD model with CSBN and CIN achieves 87.18% rank-1 accuracy and 69.98% mAP on the CS+MT+C3→M benchmark, significantly outperforming the baseline.
• Adding CFD to all four stages of the backbone yields the best performance, demonstrating the importance of deep integration.
• The ablation study confirms that domain loss improves CFD performance, with a 2.38% absolute gain in mAP when added to the CFD model.
• Joint spatial and channel attention (SC) yields the highest performance (73.11% R1, 88.87% mAP), outperforming spatial-only or channel-only attention.
• t-SNE visualization confirms that CFD reduces domain gaps and improves inter-domain compactness, especially on unseen datasets like VIPeR, PRID, GRID, and i-LIDS.
• Feature map visualization shows that CFD enhances identity-relevant features on salient body regions while preserving domain-relevant responses, illustrating the persistence of entangled features.