[论文解读] Learning to Adapt Invariance in Memory for Person Re-identification

Zhun Zhong, Liang Zheng+3 more|arXiv (Cornell University)|Aug 1, 2019

Video Surveillance and Tracking Methods参考文献 61被引用 36

一句话总结

它提出一个用于 person re-ID 的无监督域自适应框架，通过 exemplar memory 和 Graph-based Positive Prediction (GPP) 实现 exemplar、camera 和 neighborhood 不变性，以提高目标域的泛化性能。

ABSTRACT

This work considers the problem of unsupervised domain adaptation in person re-identification (re-ID), which aims to transfer knowledge from the source domain to the target domain. Existing methods are primary to reduce the inter-domain shift between the domains, which however usually overlook the relations among target samples. This paper investigates into the intra-domain variations of the target domain and proposes a novel adaptation framework w.r.t. three types of underlying invariance, i.e., Exemplar-Invariance, Camera-Invariance, and Neighborhood-Invariance. Specifically, an exemplar memory is introduced to store features of samples, which can effectively and efficiently enforce the invariance constraints over the global dataset. We further present the Graph-based Positive Prediction (GPP) method to explore reliable neighbors for the target domain, which is built upon the memory and is trained on the source samples. Experiments demonstrate that 1) the three invariance properties are indispensable for effective domain adaptation, 2) the memory plays a key role in implementing invariance learning and improves the performance with limited extra computation cost, 3) GPP could facilitate the invariance learning and thus significantly improves the results, and 4) our approach produces new state-of-the-art adaptation accuracy on three re-ID large-scale benchmarks.

研究动机与目标

• 研究目标域内部变异如何影响无监督域自适应的 person re-ID。
• 引入三种目标不变性属性——Exemplar-Invariance、Camera-Invariance 和 Neighborhood-Invariance，并评估它们的影响。
• 开发一个基于内存的框架，在整个训练集上强制执行这些不变性，且额外计算开销尽量小。
• 整合 Graph-based Positive Prediction (GPP) 模块，以可靠地识别用于不变学习的正邻居。

提出的方法

• 使用一个双分支训练框架，包含共享特征提取器和领域特定的 exemplar memories（source 和 target）。
• 在目标分支上施加三种不变性损失：exemplar-invariance（每个目标 exemplar 分类到其自身类别）、camera-invariance（目标图像及 CamStyle 变体共用一个类别）、以及 neighborhood-invariance（目标图像向可靠邻居进行分类）。
• 引入一个 exemplar memory，存储最新的目标特征，并使全局、非批次学习的不变性成为可能。
• 提出 Graph-based Positive Prediction (GPP) 通过从 top-k memory 邻居构建图并应用图卷积网络来预测正邻居，从而细化邻居关系（在 source 数据上训练，在 target 上应用）。
• 在 source 域上训练 GPP，以学习正邻居预测，然后应用于 target 域以帮助 NI 学习，而不通过特征提取器进行反向传播。
• 利用 CamStyle（CycleGAN/StarGAN）为目标图像生成相机风格变体，以支持 CI。

实验结果

研究问题

• RQ1 Can incorporating exemplar-, camera-, and neighborhood-invariance improve unsupervised domain adaptation for person re-ID compared to inter-domain distribution alignment alone?
• RQ2 Does an exemplar memory enable effective global (not mini-batch) invariance learning with minimal extra computation?
• RQ3 Does Graph-based Positive Prediction enhance the reliability of neighborhood-invariance learning for target domain adaptation?
• RQ4 How does the proposed method perform on large-scale re-ID benchmarks (Market-1501, DukeMTMC-reID, MSMT17) compared with state-of-the-art UDA approaches?

主要发现

• Three invariance properties are indispensable for effective domain adaptation in re-ID.
• Memory-based invariance learning significantly improves performance with limited additional computation and memory.
• GPP improves invariance learning by providing more reliable positive neighbors for NI, leading to better accuracy (notably mAP) on target data.
• The approach achieves state-of-the-art adaptation accuracy on three large-scale re-ID benchmarks (Market-1501, DukeMTMC-reID, MSMT17).