[Paper Review] Dual-Refinement: Joint Label and Feature Refinement for Unsupervised Domain Adaptive Person Re-Identification

Yongxing Dai, Jun Liu+3 more|arXiv (Cornell University)|Dec 26, 2020

Video Surveillance and Tracking Methods102 references102 citations

TL;DR

This paper proposes Dual-Refinement, a novel unsupervised domain adaptation method for person re-identification that jointly refines pseudo labels during off-line clustering and features during on-line training. By introducing hierarchical clustering with representative prototypes and an instant memory spread-out regularization, the method reduces noisy label influence and improves feature discriminability, achieving state-of-the-art performance with minimal computational overhead on standard benchmarks.

ABSTRACT

Unsupervised domain adaptive (UDA) person re-identification (re-ID) is a challenging task due to the missing of labels for the target domain data. To handle this problem, some recent works adopt clustering algorithms to off-line generate pseudo labels, which can then be used as the supervision signal for on-line feature learning in the target domain. However, the off-line generated labels often contain lots of noise that significantly hinders the discriminability of the on-line learned features, and thus limits the final UDA re-ID performance. To this end, we propose a novel approach, called Dual-Refinement, that jointly refines pseudo labels at the off-line clustering phase and features at the on-line training phase, to alternatively boost the label purity and feature discriminability in the target domain for more reliable re-ID. Specifically, at the off-line phase, a new hierarchical clustering scheme is proposed, which selects representative prototypes for every coarse cluster. Thus, labels can be effectively refined by using the inherent hierarchical information of person images. Besides, at the on-line phase, we propose an instant memory spread-out (IM-spread-out) regularization, that takes advantage of the proposed instant memory bank to store sample features of the entire dataset and enable spread-out feature learning over the entire training data instantly. Our Dual-Refinement method reduces the influence of noisy labels and refines the learned features within the alternative training process. Experiments demonstrate that our method outperforms the state-of-the-art methods by a large margin.

Motivation & Objective

• Address the challenge of noisy pseudo labels in unsupervised domain adaptive person re-identification (UDA-ReID), where off-line clustering generates unreliable labels due to high intra-cluster variance.
• Improve feature discriminability in the target domain by mitigating the negative impact of noisy pseudo labels during on-line training.
• Develop a dual optimization framework that alternately refines pseudo labels and features to break the feedback loop of error propagation between label quality and feature learning.
• Reduce computational cost while maintaining high performance by introducing an efficient instant memory bank for global feature spread-out regularization.

Proposed method

• Propose a two-stage hierarchical clustering scheme: first perform coarse clustering on target domain features, then apply fine clustering within each coarse cluster to identify representative prototypes.
• Use the centroids of fine clusters as representative prototypes for each coarse cluster, enabling global, robust similarity computation between samples and clusters instead of relying on local neighborhood similarities.
• Introduce an instant memory spread-out (IM-spread-out) regularization that leverages an instant memory bank storing all target domain features to enforce feature spread-out across the entire dataset during on-line training.
• Formulate the IM-spread-out loss as a contrastive regularization that encourages hard noisy samples to be pushed away from decision boundaries while preserving class consistency.
• Combine the hierarchical clustering-based pseudo label refinement with the IM-spread-out regularization in an alternating training loop to jointly optimize label purity and feature discriminability.
• Design a loss function that balances metric learning under both coarse and fine pseudo labels using a learnable weight α, and integrate the IM-spread-out regularization with a learnable weight µ.

Experimental results

Research questions

• RQ1 How can we improve the reliability of off-line pseudo labels in UDA-ReID by leveraging hierarchical structure in target domain data?
• RQ2 To what extent can on-line feature learning be improved by mitigating the influence of noisy pseudo labels through global feature regularization?
• RQ3 Can an instant memory bank enable effective, low-cost global feature spread-out regularization without significant computational overhead?
• RQ4 How do hyperparameters such as the number of fine clusters and the loss weight for spread-out regularization affect the final performance?

Key findings

• Dual-Refinement achieves a rank-1 accuracy of 82.1% on Market1501 and 82.1% on DukeMTMC, significantly outperforming the baseline method (72.5%) and state-of-the-art methods like MMT (78.0%).
• The method improves mAP from 3.17% (baseline) to 3.53% on Market1501 and from 11.45% to 15.07% on DukeMTMC, demonstrating consistent gains across metrics.
• The ablation study shows that the optimal loss weight α = 0.5 and µ = 0.1, with performance peaking at these values and degrading when increased further, indicating a trade-off between label refinement and regularization strength.
• The number of k−nearest neighbors (k) has a strong impact: performance remains stable for k ≤ 10 but drops sharply when k > 10, suggesting that too many noisy positives degrade the spread-out effect.
• The fine cluster number R is critical: R = 5 for Duke→Market1501 and R = 2 for Market1501→Duke achieves the best performance, showing that hierarchical clustering enhances label reliability.
• The method achieves a 10% improvement in mAP over the baseline with only 0.36 hours of extra training time, demonstrating high efficiency and scalability.