[Paper Review] Structured Consistency Loss for semi-supervised semantic segmentation

JongMok Kim, Joo Young Jang+2 more|arXiv (Cornell University)|Jan 14, 2020

Domain Adaptation and Few-Shot LearningComputer Science31 references47 citations

TL;DR

The paper introduces a structured consistency loss that accounts for inter-pixel correlations to enhance semi-supervised semantic segmentation and improves efficiency with CutMix, achieving top results on Cityscapes benchmarks.

ABSTRACT

The consistency loss has played a key role in solving problems in recent studies on semi-supervised learning. Yet extant studies with the consistency loss are limited to its application to classification tasks; extant studies on semi-supervised semantic segmentation rely on pixel-wise classification, which does not reflect the structured nature of characteristics in prediction. We propose a structured consistency loss to address this limitation of extant studies. Structured consistency loss promotes consistency in inter-pixel similarity between teacher and student networks. Specifically, collaboration with CutMix optimizes the efficient performance of semi-supervised semantic segmentation with structured consistency loss by reducing computational burden dramatically. The superiority of proposed method is verified with the Cityscapes; The Cityscapes benchmark results with validation and with test data are 81.9 mIoU and 83.84 mIoU respectively. This ranks the first place on the pixel-level semantic labeling task of Cityscapes benchmark suite. To the best of our knowledge, we are the first to present the superiority of state-of-the-art semi-supervised learning in semantic segmentation.

Motivation & Objective

• Motivate semi-supervised semantic segmentation by leveraging consistency loss with inter-pixel correlations.
• Introduce a structured consistency loss to improve computational efficiency in pixel-wise segmentation.
• Demonstrate state-of-the-art performance on Cityscapes validation and test sets.
• Show that the approach ranks first on pixel-level semantic labeling in the Cityscapes benchmark.

Proposed method

• Propose a structured consistency loss that models inter-pixel correlations rather than only matched pixel pairs.
• Integrate the loss with CutMix to reduce computational complexity.
• Aim to improve efficiency and effectiveness of semi-supervised learning for semantic segmentation.
• Demonstrate empirical gains on Cityscapes benchmark data (validation and test) with reported mIoU scores.

Experimental results

Research questions

• RQ1 Can a structured consistency loss that captures inter-pixel relationships improve semi-supervised semantic segmentation performance?
• RQ2 Does incorporating the proposed loss with CutMix reduce computational complexity while maintaining or improving accuracy?
• RQ3 What is the impact of the proposed method on Cityscapes benchmark metrics in semi-supervised settings?

Key findings

• Achieves 81.9 mIoU on Cityscapes validation data.
• Achieves 83.84 mIoU on Cityscapes test data.