[论文解读] Cross-domain Human Parsing via Adversarial Feature and Label Adaptation

Si Liu, Yao Sun+5 more|arXiv (Cornell University)|Jan 4, 2018

Multimodal Machine Learning Applications参考文献 35被引用 13

一句话总结

该论文提出了一种跨域人体解析框架，通过对抗性特征和结构化标签适应，将预训练的解析器从源域（例如 LIP 数据集）适应到目标域，而无需人工标注。该方法联合最小化特征中的域差异并保持高阶标签关系，实现了最先进性能，F-1 分数相比基线方法最高提升 2%。

ABSTRACT

Human parsing has been extensively studied recently due to its wide applications in many important scenarios. Mainstream fashion parsing models focus on parsing the high-resolution and clean images. However, directly applying the parsers trained on benchmarks to a particular application scenario in the wild, e.g., a canteen, airport or workplace, often gives non-satisfactory performance due to domain shift. In this paper, we explore a new and challenging cross-domain human parsing problem: taking the benchmark dataset with extensive pixel-wise labeling as the source domain, how to obtain a satisfactory parser on a new target domain without requiring any additional manual labeling? To this end, we propose a novel and efficient cross-domain human parsing model to bridge the cross-domain differences in terms of visual appearance and environment conditions and fully exploit commonalities across domains. Our proposed model explicitly learns a feature compensation network, which is specialized for mitigating the cross-domain differences. A discriminative feature adversarial network is introduced to supervise the feature compensation to effectively reduce the discrepancy between feature distributions of two domains. Besides, our model also introduces a structured label adversarial network to guide the parsing results of the target domain to follow the high-order relationships of the structured labels shared across domains. The proposed framework is end-to-end trainable, practical and scalable in real applications. Extensive experiments are conducted where LIP dataset is the source domain and 4 different datasets including surveillance videos, movies and runway shows are evaluated as target domains. The results consistently confirm data efficiency and performance advantages of the proposed method for the cross-domain human parsing problem.

研究动机与目标

• 解决在将基于高质量基准数据集训练的模型应用于餐厅、机场等现实场景时，人体解析中的域偏移问题。
• 开发一种方法，实现从具有丰富标注的源域到无标注目标域的有效适应。
• 在最小化域间视觉与环境差异的同时，保持跨域的结构化标签一致性。
• 为现实世界中人体解析模型的部署，提供端到端、可扩展且实用的解决方案。

提出的方法

• 引入可学习的特征补偿网络，将源域和目标域的特征对齐至共享潜在空间。
• 采用判别式特征对抗网络，通过训练判别器区分源域和目标域的特征表示，以减少域间差异。
• 引入结构化标签对抗网络，引导目标域预测遵循跨域共享的高阶标签关系（例如，头部在躯干之上）。
• 通过对抗性目标联合训练特征补偿与标签适应模块，实现端到端优化。
• 采用多尺度监督策略，利用 pool5 层的特征以增强特征表示质量。
• 在标签一致性损失中使用常数 K_C = 5，以稳定训练并提升泛化能力。

实验结果

研究问题

• RQ1 对抗性特征适应是否能在无需人工标注的情况下有效减少跨域人体解析中的域偏移？
• RQ2 结构化标签对抗训练是否能有效保留跨域的高阶语义关系（例如，身体部位的层级结构）？
• RQ3 联合特征与标签适应相较于单模态适应或域不变基线方法有何优势？
• RQ4 该模型在低分辨率、低光照或高度多变的真实场景中具有多大程度的泛化能力？

主要发现

• 所提方法通过结合特征与标签适应，在 PridA 数据集上实现了平均 92.83% 的 F-1 分数，相比 'Source Only' 基线提升 1.05%，相比 DANN 提升 0.82%。
• 在 Daily Video 数据集上，该方法实现了 87.51% 的 F-1 分数，超过 'Target Only' 上限，表明即使在训练数据有限的情况下，仍能实现有效的知识迁移。
• 在 PridB 数据集上，该方法在 'U-body' 类别上实现了 78.27% 的 F-1 分数，相比 'Source Only'（74.56%）和 'DANN'（74.79%）提升超过 3.5 个百分点。
• 'Feat. + Lab. Adapt' 变体始终优于单独的 'Feat. Adapt' 或 'Lab. Adapt'，证明了联合优化的互补优势。
• 定性结果表明，左/右肢体和头发的预测精度显著提升，尤其在低光照和高动态场景（如室内餐厅和监控视频）中表现更优。
• 该模型能有效处理部分遮挡、运动模糊和低分辨率输入等挑战性情况，而基线模型在这些情况下往往失效。