[论文解读] Self-paced Contrastive Learning with Hybrid Memory for Domain Adaptive Object Re-ID

Yixiao Ge, Feng Zhu+3 more|arXiv (Cornell University)|Jun 4, 2020

Video Surveillance and Tracking Methods参考文献 68被引用 342

一句话总结

该论文 introduces a self-paced contrastive learning framework with a hybrid memory that jointly uses source-class centroids, target-cluster centroids, and un-clustered target instances to improve domain adaptive object re-ID, with a reliability-driven self-paced strategy to refine learning targets.

ABSTRACT

Domain adaptive object re-ID aims to transfer the learned knowledge from the labeled source domain to the unlabeled target domain to tackle the open-class re-identification problems. Although state-of-the-art pseudo-label-based methods have achieved great success, they did not make full use of all valuable information because of the domain gap and unsatisfying clustering performance. To solve these problems, we propose a novel self-paced contrastive learning framework with hybrid memory. The hybrid memory dynamically generates source-domain class-level, target-domain cluster-level and un-clustered instance-level supervisory signals for learning feature representations. Different from the conventional contrastive learning strategy, the proposed framework jointly distinguishes source-domain classes, and target-domain clusters and un-clustered instances. Most importantly, the proposed self-paced method gradually creates more reliable clusters to refine the hybrid memory and learning targets, and is shown to be the key to our outstanding performance. Our method outperforms state-of-the-arts on multiple domain adaptation tasks of object re-ID and even boosts the performance on the source domain without any extra annotations. Our generalized version on unsupervised object re-ID surpasses state-of-the-art algorithms by considerable 16.7% and 7.9% on Market-1501 and MSMT17 benchmarks.

研究动机与目标

• 激励对象重识别的无监督域自适应，并解决基于聚类的伪标签中对源数据利用不足及异常值的问题。
• 提出一个统一的对比学习框架，利用源类别、目标簇以及未聚簇实例信号，在混合记忆中融合。
• 引入一种带有聚类可靠性准则的自定步伐学习策略，以逐步形成可靠的目标簇。
• 表明该方法在跨域性能上有所提升，并且在无需额外标注的情况下也能提升源域性能。

提出的方法

• 引入一个混合记忆，存储源域类别质心、目标域簇质心以及目标域未聚簇实例的特征。
• 定义一个统一的对比损失，将源类别、目标簇和未聚簇实例视为独立的类别（Eq. 1）。
• 使用动态、非参数的记忆，通过动量更新质心与实例特征（Eq. 3 和 Eq. 4）。
• 从初始特征初始化目标域簇，并在学习过程中更新簇（Eq. 2）。
• 应用一个自定步伐聚类可靠性准则，结合独立性与紧凑性来保留可靠簇并重新分类不可靠的簇（Eq. 5, Eq. 6）。

实验结果

研究问题

• RQ1 结合源标签和目标域结构的统一对比目标是否能提升域自适应的重识别？
• RQ2 捕捉类级、簇级和实例级信号的混合记忆是否能提升跨域的特征学习？
• RQ3 基于簇可靠性的自定步伐策略是否能防止嘈杂的伪标签降低学习效果？
• RQ4 开发利用目标域未聚簇实例在多大程度上能够提升目标域，甚至源域的性能？

主要发现

• 该方法在多个对象重识别的域自适应任务上，使用简单的 ResNet-50 骨干网络，超越了最先进的UDA方法，获得最高5.0%的mAP提升。
• 在未使用带标签的源数据的无监督版本，在 Market-1501 上显著超过此前方法16.7%的mAP，在 MSMT17 基准上超越7.9%的mAP。
• 通过与未标注的目标域数据联合训练，该框架可将源域性能提升至多6.6% 的mAP。
• 统一的对比学习结合混合记忆和自定步伐的可靠性带来显著的消融增益；消融分析显示包括源类别、目标簇和未聚簇实例信号以及自定步伐可靠性准则的重要性。
• 该方法在真实到真实和合成到真实的数据集上，对于人员和车辆的重识别，展示出竞争力或优越的性能。