[论文解读] Semi-Supervised Semantic Segmentation via Gentle Teaching Assistant

Ying Jin, Jiaqi Wang+1 more|arXiv (Cornell University)|Jan 18, 2023

Advanced Neural Network Applications被引用 31

一句话总结

引入 GTA-Seg，一种带有 Gentle Teaching Assistant 的师生框架，它从伪标签中学习以提升特征表示，同时在掩码预测器中避免不可靠的伪标签，在标准基准测试中实现最先进的结果。

ABSTRACT

Semi-Supervised Semantic Segmentation aims at training the segmentation model with limited labeled data and a large amount of unlabeled data. To effectively leverage the unlabeled data, pseudo labeling, along with the teacher-student framework, is widely adopted in semi-supervised semantic segmentation. Though proved to be effective, this paradigm suffers from incorrect pseudo labels which inevitably exist and are taken as auxiliary training data. To alleviate the negative impact of incorrect pseudo labels, we delve into the current Semi-Supervised Semantic Segmentation frameworks. We argue that the unlabeled data with pseudo labels can facilitate the learning of representative features in the feature extractor, but it is unreliable to supervise the mask predictor. Motivated by this consideration, we propose a novel framework, Gentle Teaching Assistant (GTA-Seg) to disentangle the effects of pseudo labels on feature extractor and mask predictor of the student model. Specifically, in addition to the original teacher-student framework, our method introduces a teaching assistant network which directly learns from pseudo labels generated by the teacher network. The gentle teaching assistant (GTA) is coined gentle since it only transfers the beneficial feature representation knowledge in the feature extractor to the student model in an Exponential Moving Average (EMA) manner, protecting the student model from the negative influences caused by unreliable pseudo labels in the mask predictor. The student model is also supervised by reliable labeled data to train an accurate mask predictor, further facilitating feature representation. Extensive experiment results on benchmark datasets validate that our method shows competitive performance against previous methods. Code is available at https://github.com/Jin-Ying/GTA-Seg.

研究动机与目标

• 推动在半监督语义分割中更充分利用未标注数据。
• 区分伪标签如何影响特征提取器与掩码预测器。
• 利用 Gentle Teaching Assistant 仅通过 EMA 传输有益的表示知识。
• 通过再加权机制提升对嘈杂伪标签的鲁棒性。
• 在 PASCAL VOC 2012 和 Cityscapes 数据集上展示具有竞争力的提升。

提出的方法

• 在教师-学生框架中增加一个额外的 Gentle Teaching Assistant (GTA) 模块。
• 由教师生成伪标签，并通过像素置信度和拉普拉斯平滑进行再加权。
• 在未标注数据上训练 GTA，并通过 EMA 将仅特征提取器的知识传递给学生。
• 用真实标签监督学生以训练掩码预测器，同时通过来自学生的 EMA 更新教师。

实验结果

研究问题

• RQ1 伪标签是否能够在不污染掩码预测器的前提下改善特征表示？
• RQ2 通过 GTA 将表示学习与掩码预测分离是否能提升半监督分割的性能？
• RQ3 对伪标签进行再加权如何影响分割质量以及对误校准的鲁棒性？

主要发现

• GTA-Seg 在不同数据规模下均优于原生的教师-学生基线。
• 在 PASCAL VOC 2012（原始训练集）上，GTA-Seg 在有标签数据比例范围内实现 70.02 到 80.47 mIoU，超过之前的方法。
• 在与 GTA-Seg 联用时，重新加权的伪标签提供了额外的提升。
• 消融研究显示，Gentle Teaching Assistant 与表示传输对提升贡献最大，超越再加权的影响。
• 可视化结果表明，在 GTA-Seg 下轮廓更清晰、类别分割更准确。