[论文解读] Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation

Qiming Zhang, Jing Zhang+2 more|arXiv (Cornell University)|Oct 29, 2019

Domain Adaptation and Few-Shot Learning被引用 97

一句话总结

本文提出了一种以类别锚点为引导的UDA模型（CAG-UDA），利用源域的类别质心作为锚点来识别活跃的目标样本、生成伪标签，并进行跨域语义分割的类别级特征对齐。

ABSTRACT

Unsupervised domain adaptation (UDA) aims to enhance the generalization capability of a certain model from a source domain to a target domain. UDA is of particular significance since no extra effort is devoted to annotating target domain samples. However, the different data distributions in the two domains, or \emph{domain shift/discrepancy}, inevitably compromise the UDA performance. Although there has been a progress in matching the marginal distributions between two domains, the classifier favors the source domain features and makes incorrect predictions on the target domain due to category-agnostic feature alignment. In this paper, we propose a novel category anchor-guided (CAG) UDA model for semantic segmentation, which explicitly enforces category-aware feature alignment to learn shared discriminative features and classifiers simultaneously. First, the category-wise centroids of the source domain features are used as guided anchors to identify the active features in the target domain and also assign them pseudo-labels. Then, we leverage an anchor-based pixel-level distance loss and a discriminative loss to drive the intra-category features closer and the inter-category features further apart, respectively. Finally, we devise a stagewise training mechanism to reduce the error accumulation and adapt the proposed model progressively. Experiments on both the GTA5$ ightarrow $Cityscapes and SYNTHIA$ ightarrow $Cityscapes scenarios demonstrate the superiority of our CAG-UDA model over the state-of-the-art methods. The code is available at \url{https://github.com/RogerZhangzz/CAG_UDA}.

研究动机与目标

• 在类别层级错位和域迁移的情形下，推动语义分割的无监督域自适应。
• 引入类别锚点，以在目标域实现类别层面的特征对齐和可靠的伪标签。
• 开发一种分阶段训练程序，以减少来自伪标签的错误累积。
• 在 GTA5→Cityscapes 和 SYNTHIA→Cityscapes 数据集上展示最先进的性能。

提出的方法

• 从编码器的特征计算源域的类别层面特征质心（类别锚点）。
• 通过度量与锚点的距离来识别活跃的目标样本，并为这些样本选择伪标签。
• 基于最近的锚点并使用边界条件给活跃目标样本分配伪标签。
• 使用按类别的距离损失将活跃目标特征与锚点对齐，并对伪标签使用交叉熵损失。
• 可选地将基于概率的伪标签与基于锚点的伪标签结合，以加强监督。
• 阶段性训练（分阶段训练）以减轻伪标签累积的错误。

实验结果

研究问题

• RQ1 基于类别锚点引导的特征对齐是否能够改善类别分离并降低语义分割中的域偏移？
• RQ2 基于类别锚点的活跃目标样本伪标签是否比基于概率的伪标签提供更可靠的监督？
• RQ3 分阶段训练策略是否能减少错误累积并提升在 GTA5→Cityscapes 与 SYNTHIA→Cityscapes 上的最终分割性能？

主要发现

road	sidewalk	building	wall	fence	pole	light	sign	vege.	terrace	sky	person	rider	car	truck	bus	train	motor	bike	mIoU
93.2	57.0	85.6	35.7	25.1	37.5	30.8	45.3	87.1	50.1	89.4	62.7	40.8	87.8	18.0	32.4	34.5	34.4	35.4	51.7

• CAG-UDA 在 GTA5→Cityscapes 的阶段性训练中达到 50.2 mIoU，超越了最先进的方法。
• 在 Cityscapes 的验证/测试集上，CAG-UDA 达到 51.7 mIoU，显示出良好的泛化能力。
• CAG-UDA 在依赖隐式类别层对齐的方法上持续提升性能，特别是在像杆子和标志等小物体上。
• 在存在域移位的情况下，基于锚点的伪标签提供比基于概率的伪标签更可靠的监督。
• 分阶段训练使学习更稳定并缓解伪标签的错误累积。