[论文解读] Domain Adaptation for Semantic Segmentation with Maximum Squares Loss

Minghao Chen, Hongyang Xue+1 more|arXiv (Cornell University)|Sep 30, 2019

Domain Adaptation and Few-Shot Learning参考文献 34被引用 39

一句话总结

本文提出最大平方损失，用于解决语义分割无监督域适应中的概率不平衡问题，提升目标域训练，使困难样本获得有意义的梯度，而不使用对抗判别器。

ABSTRACT

Deep neural networks for semantic segmentation always require a large number of samples with pixel-level labels, which becomes the major difficulty in their real-world applications. To reduce the labeling cost, unsupervised domain adaptation (UDA) approaches are proposed to transfer knowledge from labeled synthesized datasets to unlabeled real-world datasets. Recently, some semi-supervised learning methods have been applied to UDA and achieved state-of-the-art performance. One of the most popular approaches in semi-supervised learning is the entropy minimization method. However, when applying the entropy minimization to UDA for semantic segmentation, the gradient of the entropy is biased towards samples that are easy to transfer. To balance the gradient of well-classified target samples, we propose the maximum squares loss. Our maximum squares loss prevents the training process being dominated by easy-to-transfer samples in the target domain. Besides, we introduce the image-wise weighting ratio to alleviate the class imbalance in the unlabeled target domain. Both synthetic-to-real and cross-city adaptation experiments demonstrate the effectiveness of our proposed approach. The code is released at https://github. com/ZJULearning/MaxSquareLoss.

研究动机与目标

• 通过实现从合成数据到真实数据的语义分割无监督域适应，降低标注成本的动机。
• 指出UDA中熵最小化的局限性，因为梯度偏向于易于迁移的样本。
• 提出最大平方损失以平衡目标域的梯度并改善类别层面的对齐。
• 通过图像级加权方案和多层次自产引导来解决目标域类别不平衡，提升低层特征。

提出的方法

• 将最大平方损失定义为目标损失项：L_T(x_t) = -(1/2N) sum_n sum_c (p_t^{n,c})^2，即产生线性增长的梯度以平衡易/难样本。
• 证明最小化最大平方损失等价于最大化目标预测与均匀分布之间的皮尔逊卡方散度，从而促进类别层面的分布对齐。
• 引入图像级加权因子以处理未标记目标域中的类别不平衡，基于每张图像的预测类别存在情况计算。
• 通过结合高级别与低级别输出来产生低级特征学习的伪引导（自产引导），整合多层次自产引导。
• 提供实际的训练细节和超参数，使用基于 ResNet 的骨干网和 Deeplabv2 风格的架构，带有或不带有图像级加权及多级引导。

Figure 1 : In UDA, the gradient of the entropy minimization method ( $H$ ) is focused on well-classified samples in the target domain. Consequently, we propose the maximum squares loss ( $MS$ ), which is the negative sum of squared probabilities. The gradient of the maximum squares loss is linearly

实验结果

研究问题

• RQ1 最大平方损失是否能够通过在目标类别之间平衡梯度来缓解语义分割UDA中熵最小化的偏倚？
• RQ2 是否，通过图像级别的类别平衡加权方案，在没有目标标签的情况下改善对目标域类别不平衡的处理？
• RQ3 多层次自产引导技术是否通过使用更高层次的预测来引导低层次特征学习，从而进一步提升性能？

主要发现

• 最大平方损失在目标类别之间平衡梯度贡献，并相较于熵最小化提升困难样本的训练效率。
• 优化最大平方损失等价于最大化到均匀分布的皮尔逊卡方散度，促进跨域的类别层面分布对齐。
• 图像级别的类别平衡加权在语义分割的UDA中显著提升性能，相较于传统的全局类别加权。
• 增加多层次自产引导通过利用高层次预测来引导低层次特征学习，带来显著的性能提升。
• 在从合成到真实（GTA5/SYNTHIA 到 Cityscapes）和跨城市适应中，基于 MaxSquare 的方法在不需要判别器的情况下实现具有竞争力或最先进的结果。

Figure 2 : From GTA5 to Cityscapes, the mean of prediction probability v.s. Intersection over Union(IoU) for each target class. They are almost linearly related. Thus well-classified classes (high IoU) have larger prediction probability.