[Paper Review] Semi-Supervised Semantic Segmentation via Dynamic Self-Training and Class-Balanced Curriculum

Zhengyang Feng, Qianyu Zhou+4 more|arXiv (Cornell University)|Apr 18, 2020

Domain Adaptation and Few-Shot Learning57 references55 citations

TL;DR

This paper proposes Dynamic Self-Training and Class-Balanced Curriculum (DST-CBC), a novel semi-supervised semantic segmentation method that leverages inter-model disagreement via prediction confidence to build a dynamic, noise-robust loss, enabling efficient and thorough utilization of all unlabeled data. Without adversarial training or architectural changes, DST-CBC outperforms prior methods across all labeled data ratios on multiple datasets.

ABSTRACT

In this work, we propose a novel and concise approach for semi-supervised semantic segmentation. The major challenge of this task lies in how to exploit unlabeled data efficiently and thoroughly. Previous state-of-the-art methods utilize unlabeled data by GAN-based self-training or consistency regularization. However, these methods either suffer from noisy self-supervision and class-imbalance, resulting in a low unlabeled data utilization rate, or do not consider the apparent link between self-training and consistency regularization. Our method, Dynamic Self-Training and Class-Balanced Curriculum (DST-CBC), exploits inter-model disagreement by prediction confidence to construct a dynamic loss robust against pseudo label noise, enabling it to extend pseudo labeling to a class-balanced curriculum learning process. While we further show that our method implicitly includes consistency regularization. Thus, DST-CBC not only exploits unlabeled data efficiently, but also thoroughly utilizes $all$ unlabeled data. Without using adversarial training or any kind of modification to the network architecture, DST-CBC outperforms existing methods on different datasets across all labeled ratios, bringing semi-supervised learning yet another step closer to match the performance of fully-supervised learning for semantic segmentation. Our code and data splits are available at: this https URL .

Motivation & Objective

• To address the low utilization of unlabeled data in semi-supervised semantic segmentation due to noisy pseudo labels and class imbalance.
• To bridge the gap between self-training and consistency regularization by implicitly incorporating the latter through dynamic loss construction.
• To develop a method that efficiently utilizes all unlabeled data without requiring architectural modifications or adversarial training.
• To improve performance across varying labeled data ratios, bringing semi-supervised learning closer to fully supervised performance.

Proposed method

• DST-CBC constructs a dynamic loss based on inter-model disagreement and prediction confidence to reduce noise in pseudo labels.
• It introduces a class-balanced curriculum learning process that prioritizes reliable pseudo-labeled samples per class, improving training stability.
• The method implicitly incorporates consistency regularization by leveraging model agreement across forward passes, enhancing robustness.
• It avoids adversarial training and maintains a standard network architecture, ensuring compatibility and simplicity.
• The dynamic loss is updated iteratively during training, adapting to the confidence and disagreement of predictions on unlabeled data.
• The framework enables full utilization of all unlabeled samples by progressively refining pseudo-label quality through confidence-based filtering.

Experimental results

Research questions

• RQ1 How can unlabeled data be efficiently and thoroughly utilized in semi-supervised semantic segmentation without introducing noisy pseudo labels?
• RQ2 What is the relationship between self-training and consistency regularization, and can it be leveraged implicitly in a unified framework?
• RQ3 Can a class-balanced curriculum learning strategy improve model performance when combined with dynamic self-training?
• RQ4 How does inter-model disagreement, measured via prediction confidence, enhance robustness to noisy pseudo labels?
• RQ5 Can a method without adversarial training or architectural modifications outperform existing state-of-the-art approaches?

Key findings

• DST-CBC achieves state-of-the-art performance on multiple benchmark datasets across all labeled data ratios, demonstrating consistent improvement over prior methods.
• The method outperforms existing approaches without using adversarial training or modifying the network architecture.
• By leveraging inter-model disagreement and confidence, DST-CBC significantly reduces the impact of noisy pseudo labels during self-training.
• The class-balanced curriculum learning process ensures that underrepresented classes are not neglected during training.
• The framework implicitly incorporates consistency regularization, enhancing model robustness without explicit design.
• All unlabeled data are effectively utilized, as evidenced by the method's ability to scale performance across varying labeled data ratios.