[Paper Review] Benchmarks for Corruption Invariant Person Re-identification

Ming‐Hui Chen, Zhiqiang Wang+1 more|arXiv (Cornell University)|Nov 1, 2021

Retinal Imaging and Analysis18 citations

TL;DR

This paper introduces the first comprehensive benchmarks for corruption-invariant person re-identification across five single- and cross-modality datasets (Market-1501, CUHK03, MSMT17, RegDB, SYSU-MM01), applying 20 corruption types at five severity levels. It evaluates 21 state-of-the-art ReID methods and finds that transformer-based models are more robust than CNNs, random erasing harms corruption robustness, and cross-dataset generalization improves with corruption robustness, leading to a strong corruption-invariant baseline with improved data augmentation and loss design.

ABSTRACT

When deploying person re-identification (ReID) model in safety-critical applications, it is pivotal to understanding the robustness of the model against a diverse array of image corruptions. However, current evaluations of person ReID only consider the performance on clean datasets and ignore images in various corrupted scenarios. In this work, we comprehensively establish six ReID benchmarks for learning corruption invariant representation. In the field of ReID, we are the first to conduct an exhaustive study on corruption invariant learning in single- and cross-modality datasets, including Market-1501, CUHK03, MSMT17, RegDB, SYSU-MM01. After reproducing and examining the robustness performance of 21 recent ReID methods, we have some observations: 1) transformer-based models are more robust towards corrupted images, compared with CNN-based models, 2) increasing the probability of random erasing (a commonly used augmentation method) hurts model corruption robustness, 3) cross-dataset generalization improves with corruption robustness increases. By analyzing the above observations, we propose a strong baseline on both single- and cross-modality ReID datasets which achieves improved robustness against diverse corruptions. Our codes are available on https://github.com/MinghuiChen43/CIL-ReID.

Motivation & Objective

• To address the lack of systematic evaluation of person ReID models under real-world image corruptions, which are common in safety-critical surveillance applications.
• To establish the first large-scale, standardized benchmarks for corruption-invariant person re-identification across diverse single- and cross-modality datasets.
• To investigate the impact of network architecture (CNN vs. transformer) and data augmentation (e.g., random erasing) on robustness to image corruptions.
• To explore the relationship between corruption robustness and cross-dataset generalization, challenging the assumption that synthetic corruption robustness does not generalize to real-world distribution shifts.
• To propose a strong, practical baseline for corruption-invariant ReID that improves training with enhanced data augmentation and loss functions.

Proposed method

• The authors construct five corruption-invariant ReID benchmarks by applying 20 real-world corruption types (e.g., noise, blur, snow) at five severity levels to clean versions of Market-1501, CUHK03, MSMT17, RegDB, and SYSU-MM01.
• Corruption types are applied in a test-time-only manner, ensuring they are not seen during model training, enabling evaluation of true generalization to unseen corruptions.
• The evaluation is conducted in three settings: corrupted query, corrupted gallery, and corrupted query and gallery, to assess robustness under realistic failure modes.
• The authors reproduce and evaluate 21 recent state-of-the-art ReID methods, including both CNN-based and transformer-based architectures, across all benchmarks.
• A strong corruption-invariant baseline is proposed by improving random erasing, batch normalization neck (BNNeck), and identity loss, with ablation studies validating design choices.
• Activation maps and visualization techniques are used to analyze how different augmentations affect feature learning under corruption.

Experimental results

Research questions

• RQ1 How do state-of-the-art ReID models perform under diverse image corruptions, and does performance degrade significantly compared to clean settings?
• RQ2 Do transformer-based ReID models exhibit superior robustness to image corruptions compared to CNN-based models?
• RQ3 Does increasing the probability of random erasing during training improve or harm robustness to real-world image corruptions?
• RQ4 Is there a correlation between a model's robustness to synthetic corruptions and its ability to generalize across datasets with distribution shifts?
• RQ5 Can a strong, generalizable baseline be constructed that improves corruption robustness through enhanced data augmentation and loss design?

Key findings

• Transformer-based ReID models consistently outperform CNN-based models on corrupted test sets, achieving up to 10.5% higher mAP on average across corrupted benchmarks, indicating their superior ability to learn structured patterns under noise and distortion.
• Increasing the probability of random erasing during training significantly reduces model robustness on corrupted data, with mAP dropping by up to 12% on average, suggesting it hinders the model from learning discriminative features under corruption.
• Cross-dataset generalization performance improves in tandem with corruption robustness, with models achieving up to 18% higher mAP on unseen datasets when robustness to synthetic corruptions is higher, refuting prior assumptions about limited transferability.
• The proposed strong baseline, which improves random erasing, BNNeck, and identity loss, achieves state-of-the-art performance on all five benchmarks, with mAP improvements of up to 7.5% on corrupted test sets compared to standard training.
• On the SYSU-MM01 cross-modality benchmark, the best-performing method (HCTL) achieved 83.50% mAP under corrupted conditions, while the baseline improved this to 87.70% mAP, demonstrating significant robustness gains.
• The study reveals that models robust to synthetic corruptions generalize better to real-world distribution shifts, highlighting the practical value of corruption-invariant learning for real-world deployment.