[论文解读] Cloth-Changing Person Re-identification from A Single Image with Gait Prediction and Regularization

Xin Jin, Tianyu He+8 more|arXiv (Cornell University)|Mar 29, 2021

Gait Recognition and Analysis参考文献 70被引用 12

一句话总结

该论文提出GI-ReID，一种新颖的双流框架，通过从单张图像预测步态来学习对衣物变化不敏感的行人重识别表征。通过在训练过程中使用辅助步态流来正则化ReID流，而在推理时将其丢弃，该方法实现了最先进性能，在PRCC上达到37.55%的rank-1准确率，在VC-Clothes上达到93.76%，展示了在无需视频序列的情况下对衣物变化的高度鲁棒性。

ABSTRACT

Cloth-Changing person re-identification (CC-ReID) aims at matching the same person across different locations over a long-duration, e.g., over days, and therefore inevitably meets challenge of changing clothing. In this paper, we focus on handling well the CC-ReID problem under a more challenging setting, i.e., just from a single image, which enables high-efficiency and latency-free pedestrian identify for real-time surveillance applications. Specifically, we introduce Gait recognition as an auxiliary task to drive the Image ReID model to learn cloth-agnostic representations by leveraging personal unique and cloth-independent gait information, we name this framework as GI-ReID. GI-ReID adopts a two-stream architecture that consists of a image ReID-Stream and an auxiliary gait recognition stream (Gait-Stream). The Gait-Stream, that is discarded in the inference for high computational efficiency, acts as a regulator to encourage the ReID-Stream to capture cloth-invariant biometric motion features during the training. To get temporal continuous motion cues from a single image, we design a Gait Sequence Prediction (GSP) module for Gait-Stream to enrich gait information. Finally, a high-level semantics consistency over two streams is enforced for effective knowledge regularization. Experiments on multiple image-based Cloth-Changing ReID benchmarks, e.g., LTCC, PRCC, Real28, and VC-Clothes, demonstrate that GI-ReID performs favorably against the state-of-the-arts. Codes are available at https://github.com/jinx-USTC/GI-ReID.

研究动机与目标

• 为解决在长时间内个体更换衣物的行人重识别挑战，这是监控系统中的常见问题。
• 通过利用步态作为稳定且与衣物无关的生物特征线索，提升在衣物变化下的行人重识别泛化能力。
• 实现在单张图像下高效、无延迟的身份匹配，避免对视频序列或复杂数据采集的需求。
• 设计一种训练框架，利用步态作为正则化器，促使ReID模型学习到对衣物变化不变的判别性特征。
• 通过使模型对衣物变化具有鲁棒性且不牺牲推理速度，弥合最先进ReID模型与真实部署之间的差距。

提出的方法

• GI-ReID采用双流架构：主ReID-Stream用于身份匹配，辅助Gait-Stream在推理时被丢弃以提高效率。
• Gait-Stream使用步态序列预测（GSP）模块，从单张RGB图像生成合理的步态序列，以增强运动线索。
• GSP模块利用学习到的运动先验预测时间上的步态帧，使模型即使在单图像输入下也能获取动态步态特征。
• 在ReID-Stream与Gait-Stream之间施加语义一致性约束，以正则化知识蒸馏并改善特征学习。
• 框架采用端到端联合监督进行训练：ReID损失用于身份匹配，步态识别损失用于基于运动的正则化。
• 辅助步态流在训练期间作为调节器，通过强调基于步态的判别性，推动ReID流学习对衣物变化不变的特征。

Figure 1 : (a) shows a realistic wanted case that a suspect changed her coat from black to white for hiding. (b) reveals that the gait of person could help ReID, especially when the identity matching meets the cloth-changing challenge (All faces in the images are masked for anonymization).

实验结果

研究问题

• RQ1 从单张图像提取的步态特征是否能提升在显著衣物变化下的行人重识别性能？
• RQ2 如何有效利用步态作为正则化器，以促进基于图像的ReID中对衣物不敏感的表征学习？
• RQ3 是否可能仅从单张图像生成有意义的步态序列，以在无视频输入的情况下增强运动线索？
• RQ4 ReID流与步态流之间的知识蒸馏是否能在保持推理效率的同时提升对衣物变化的鲁棒性？
• RQ5 所提出的GI-ReID框架在具有长期衣物变化的基准测试中与最先进方法相比表现如何？

主要发现

• 在PRCC数据集上，GI-ReID在衣物变化条件下达到37.55%的rank-1准确率，显著优于所有基线方法，包括使用手工设计特征和深度学习模型的方法。
• 在VC-Clothes基准上，GI-ReID达到93.76%的rank-1准确率，展示了在具有挑战性的现实衣物变化场景中的强大性能。
• 即使在无衣物变化的情况下，GI-ReID在PRCC上仍达到85.97%的rank-1准确率，优于大多数现有方法，表明其强大的泛化能力。
• 消融实验确认Gait-Stream和GSP模块对性能至关重要，GSP模块的消融导致rank-1准确率下降6.5个百分点。
• 流间语义一致性约束使rank-1准确率提升3.5%，验证了其在有效知识正则化中的作用。
• 由于双流正则化机制的鲁棒性，步态预测中的失败案例（尤其是遮挡或多人群场景）并未严重降低性能。

Figure 2 : Overview of the proposed GI-ReID, which consists of ReID-Stream and Gait-Stream , they are jointly trained with a high-level Semantics Consistency (SC) constraint. The Gait-Stream plays the role of a regulator to drive ReID-Stream to learn cloth-agnostic representations from a single imag