[论文解读] Learning Disentangled Representation Implicitly via Transformer for Occluded Person Re-Identification

Mengxi Jia, Xinhua Cheng+2 more|arXiv (Cornell University)|Jul 6, 2021

Video Surveillance and Tracking Methods被引用 8

一句话总结

该论文提出DRL-Net，一种基于Transformer的无对齐框架，用于遮挡行人重识别，通过在未定义语义组件上进行全局推理，隐式地将与身份相关的特征与遮挡相关噪声分离。通过利用语义偏好对象查询以及带有去相关约束的对比特征学习模块，DRL-Net在Occluded-DukeMTMC上实现了最先进性能，显著优于先前方法。

ABSTRACT

Person re-identification (re-ID) under various occlusions has been a long-standing challenge as person images with different types of occlusions often suffer from misalignment in image matching and ranking. Most existing methods tackle this challenge by aligning spatial features of body parts according to external semantic cues or feature similarities but this alignment approach is complicated and sensitive to noises. We design DRL-Net, a disentangled representation learning network that handles occluded re-ID without requiring strict person image alignment or any additional supervision. Leveraging transformer architectures, DRL-Net achieves alignment-free re-ID via global reasoning of local features of occluded person images. It measures image similarity by automatically disentangling the representation of undefined semantic components, e.g., human body parts or obstacles, under the guidance of semantic preference object queries in the transformer. In addition, we design a decorrelation constraint in the transformer decoder and impose it over object queries for better focus on different semantic components. To better eliminate interference from occlusions, we design a contrast feature learning technique (CFL) for better separation of occlusion features and discriminative ID features. Extensive experiments over occluded and holistic re-ID benchmarks (Occluded-DukeMTMC, Market1501 and DukeMTMC) show that the DRL-Net achieves superior re-ID performance consistently and outperforms the state-of-the-art by large margins for Occluded-DukeMTMC.

研究动机与目标

• 解决在可见身体部位不一致且对齐导致性能下降的严重遮挡条件下行人重识别的挑战。
• 消除对外部监督或复杂对齐操作（如姿态估计或部件检测）的需求。
• 通过Transformer架构中的自注意力机制，隐式地将与身份相关的特征与遮挡相关噪声分离。
• 通过对比学习和去相关约束学习判别性特征，提升对遮挡干扰的鲁棒性。

提出的方法

• DRL-Net采用视觉Transformer编码器-解码器架构，对CNN提取的特征进行全局推理，实现无对齐的表征学习。
• 语义偏好对象查询引导Transformer将未定义的语义组件（如身体部位和遮挡）分解为独立的特征表示。
• 在解码器中对对象查询施加去相关约束，以强化对各个语义组件的独立关注，减少查询干扰。
• 对比特征学习（CFL）模块通过对比全局表征与局部特征，增强与ID相关和与ID无关特征之间的分离。
• 集成数据增强策略，通过在训练期间模拟多样的遮挡模式来提升鲁棒性。
• 模型联合优化ID分类损失与对比损失，其中超参数λ和α用于平衡对比损失与去相关损失的权重。

Figure 1: Illustration of the proposed DRL-Net in occluded person re-ID: Person images often suffer from various occlusions with different visible and invisible body parts which greatly complicate image alignment and similarity computation. The proposed DRL-Net is alignment-free which exploits seman

实验结果

研究问题

• RQ1 基于Transformer的架构是否能在无显式部件监督或对齐的情况下，隐式地将与身份相关的特征与遮挡噪声分离？
• RQ2 通过自注意力机制进行的全局推理，在建模遮挡行人图像中未定义语义组件之间的相互关系方面有多有效？
• RQ3 对比特征学习在多大程度上提升了遮挡特征与判别性ID特征之间的分离？
• RQ4 超参数λ（对比损失权重）和α（去相关惩罚项）如何影响模型的鲁棒性与性能？

主要发现

• DRL-Net在Occluded-DukeMTMC基准上实现了最先进性能，显著优于先前方法。
• 在Occluded-DukeMTMC上，DRL-Net实现了85.6%的Rank-1准确率和72.1%的mAP，显著超越此前最先进方法。
• 消融实验表明，增加对象查询数量（Nq）可提升性能，直至Nq=9，此后增益减小且推理成本上升。
• 模型对超参数变化具有鲁棒性：在λ=1.0和α=1.0时达到最优性能，且在一定取值范围内保持稳定。
• 可视化结果表明，与ID无关的对象查询在热力图中能自动定位遮挡区域，无需任何监督，证明了有效解耦。
• 定性结果表明，DRL-Net能正确对同一行人进行排序，而基线模型因对遮挡敏感而产生大量误报。

Figure 2: The framework of the proposed DRL-Net: DRL-Net consists of three components. The first component is occluded sample augmentation that synthesizes person images by inserting various obstacles. The second component is semantic representation extraction and disentanglement that disentangles t