[论文解读] UCC: Uncertainty guided Cross-head Co-training for Semi-Supervised Semantic Segmentation

Jiashuo Fan, Bin Gao+2 more|arXiv (Cornell University)|May 20, 2022

Advanced Neural Network Applications被引用 7

一句话总结

本文提出UCC，一种新颖的半监督语义分割框架，通过结合不确定性引导的跨头协同训练与弱增强和强增强数据增强，提升一致性与自训练效果。通过利用动态跨集复制粘贴（DCSCP）增强数据多样性，并引入不确定性引导的重加权模块（UGRM）以抑制噪声伪标签，UCC在1/16标注数据协议下实现了SOTA性能，在Cityscapes上达到77.17% mIoU，在PASCAL VOC 2012上达到76.49% mIoU。

ABSTRACT

Deep neural networks (DNNs) have witnessed great successes in semantic segmentation, which requires a large number of labeled data for training. We present a novel learning framework called Uncertainty guided Cross-head Co-training (UCC) for semi-supervised semantic segmentation. Our framework introduces weak and strong augmentations within a shared encoder to achieve co-training, which naturally combines the benefits of consistency and self-training. Every segmentation head interacts with its peers and, the weak augmentation result is used for supervising the strong. The consistency training samples' diversity can be boosted by Dynamic Cross-Set Copy-Paste (DCSCP), which also alleviates the distribution mismatch and class imbalance problems. Moreover, our proposed Uncertainty Guided Re-weight Module (UGRM) enhances the self-training pseudo labels by suppressing the effect of the low-quality pseudo labels from its peer via modeling uncertainty. Extensive experiments on Cityscapes and PASCAL VOC 2012 demonstrate the effectiveness of our UCC. Our approach significantly outperforms other state-of-the-art semi-supervised semantic segmentation methods. It achieves 77.17$\%$, 76.49$\%$ mIoU on Cityscapes and PASCAL VOC 2012 datasets respectively under 1/16 protocols, which are +10.1$\%$, +7.91$\%$ better than the supervised baseline.

研究动机与目标

• 通过结合一致性正则化与自训练的优势，解决其在半监督语义分割中的局限性。
• 通过不确定性感知的重加权机制，减少自训练过程中伪标签的噪声累积。
• 通过动态跨集复制粘贴策略，提升训练样本多样性，缓解分布差异与类别不平衡问题。
• 通过共享编码器与双增强视图，实现多个分割头之间的有效协同训练。

提出的方法

• 提出一种共享编码器的跨头协同训练框架，每个头利用弱增强与强增强生成一致的预测结果。
• 采用动态跨集复制粘贴（DCSCP）策略，通过从标注与未标注数据集中复制样本，生成多样化且受扰动的训练样本，同时保留长尾类别。
• 应用不确定性引导的重加权模块（UGRM），动态降低来自同伴头的低置信度伪标签权重，以减少自训练中的噪声。
• 使用弱增强预测结果作为强增强输出的监督信号，提升一致性与泛化能力。
• 通过可学习的超参数λ平衡有监督与无监督损失，该参数通过实验方式优化。
• 利用各头之间的共享表示，通过多视图学习提升特征紧凑性与泛化能力。

Figure 1 : Overview of our Uncertainty guided Cross-head Co-training. In our scheme, we use $head_{1}$ as an example, weakly augmented label data will flow through a shared module and corresponding segmentation $head_{1}$ to generate ${Pred}_{{1,weak}^{{}^{\prime}}}$ , then it will be supervised by

实验结果

研究问题

• RQ1 在跨头协同训练框架中，结合弱增强与强增强是否能超越标准一致性训练，进一步提升半监督语义分割性能？
• RQ2 如何在缓解分布差异与类别不平衡的同时，增强数据增强的多样性？
• RQ3 不确定性建模能否在自训练过程中有效抑制来自同伴头的噪声伪标签？
• RQ4 与传统跨模型协同训练相比，采用共享编码器的跨头协同训练是否在性能与泛化能力上更具优势？

主要发现

• 在1/16标注数据协议下，UCC在Cityscapes上达到77.17% mIoU，相比监督基线模型提升10.1%。
• 在PASCAL VOC 2012上，UCC在相同协议下实现76.49% mIoU，较监督基线提升7.91%。
• 消融实验表明，使用弱增强预测作为监督信号相比强增强监督，性能提升1.69%。
• DCSCP相比基线复制粘贴策略带来1.78%的性能增益，其中最大提升来自跨集复制与动态采样策略的结合。
• 最优的损失权衡超参数λ = 2，性能最佳；当λ = 4时，性能下降，因过度强调无监督损失。
• 跨头网络相比跨模型网络提升1.03% mIoU，证明共享表示在多头协同训练中的优势。

Figure 3 : Qualitative results on the Cityscapes val set.(a) and (b) are corresponding to target images and Ground Truth(GT), (c) represents the results of CPS [ 6 ] , (d) is our method results. Orange rectangles highlight the difference among of them.