[Paper Review] PartMix: Regularization Strategy to Learn Part Discovery for Visible-Infrared Person Re-identification

Minsu Kim, Seungryong Kim+3 more|arXiv (Cornell University)|Apr 4, 2023

Video Surveillance and Tracking Methods4 citations

TL;DR

This paper proposes PartMix, a novel data augmentation technique for visible-infrared person re-identification (VI-ReID) that synthesizes part-aware augmented samples by mixing part descriptors across modalities. By generating positive and negative samples within and across identities and applying contrastive learning with entropy-based mining, PartMix significantly improves generalization and achieves state-of-the-art performance on SYSU-MM01 and RegDB benchmarks.

ABSTRACT

Modern data augmentation using a mixture-based technique can regularize the models from overfitting to the training data in various computer vision applications, but a proper data augmentation technique tailored for the part-based Visible-Infrared person Re-IDentification (VI-ReID) models remains unexplored. In this paper, we present a novel data augmentation technique, dubbed PartMix, that synthesizes the augmented samples by mixing the part descriptors across the modalities to improve the performance of part-based VI-ReID models. Especially, we synthesize the positive and negative samples within the same and across different identities and regularize the backbone model through contrastive learning. In addition, we also present an entropy-based mining strategy to weaken the adverse impact of unreliable positive and negative samples. When incorporated into existing part-based VI-ReID model, PartMix consistently boosts the performance. We conduct experiments to demonstrate the effectiveness of our PartMix over the existing VI-ReID methods and provide ablation studies.

Motivation & Objective

• To address overfitting and modality-specific bias in part-based visible-infrared person Re-ID by introducing a tailored data augmentation strategy.
• To improve generalization by synthesizing diverse, part-aware augmented samples through inter- and intra-modality descriptor mixing.
• To reduce the adverse impact of unreliable positive and negative samples in contrastive learning via an entropy-based mining strategy.
• To enhance feature alignment and modality-invariant representation learning in part-based VI-ReID models.
• To achieve state-of-the-art performance on standard VI-ReID benchmarks through effective regularization.

Proposed method

• PartMix generates augmented samples by mixing part descriptors from visible and infrared images across identities and within the same identity.
• It creates both positive and negative samples using mixed part descriptors to encourage robust feature learning.
• Contrastive learning is applied to regularize the backbone model using the synthesized positive and negative pairs.
• An entropy-based mining strategy selects reliable positive and negative samples to reduce noise and improve training stability.
• The method is integrated into existing part-based VI-ReID models without architectural changes, enhancing generalization.
• The approach is evaluated end-to-end on SYSU-MM01 and RegDB datasets with ablation studies.

(a) MixUp [ 71 ]

Experimental results

Research questions

• RQ1 Can mixing part descriptors across visible and infrared modalities improve generalization in person Re-ID?
• RQ2 How does PartMix compare to global and local image mixture methods like MixUp and CutMix in part-based VI-ReID?
• RQ3 To what extent does entropy-based mining improve the reliability of positive and negative samples in contrastive learning for VI-ReID?
• RQ4 Does PartMix effectively reduce overfitting to specific body parts or modalities in part-based Re-ID models?
• RQ5 Can PartMix achieve state-of-the-art performance on standard VI-ReID benchmarks without architectural modifications?

Key findings

• PartMix achieves a Rank-1 accuracy of 77.78% and mAP of 74.62% on the SYSU-MM01 dataset in single-shot all-search mode, outperforming the previous SOTA MSCLNet by 0.79% in Rank-1 and 2.98% in mAP.
• On SYSU-MM01 indoor-search, PartMix improves Rank-1 accuracy by 3.03% and mAP by 3.21% over MSCLNet.
• On RegDB, PartMix achieves 84.93% Rank-1 and 82.52% mAP in infrared-to-visible mode, surpassing MSCLNet by 1.07% in Rank-1 and 4.21% in mAP.
• In visible-to-infrared mode on RegDB, PartMix achieves 85.66% Rank-1 and 82.27% mAP, outperforming MSCLNet by 1.49% in Rank-1 and 1.28% in mAP.
• PartMix significantly outperforms MixUp, Manifold MixUp, and CutMix, with a 5.08% mAP gain over CutMix on SYSU-MM01 multi-shot all-search.
• Ablation studies confirm the effectiveness of both the part descriptor mixing strategy and the entropy-based mining component in improving performance.

(b) CutMix [ 70 ]