[Paper Review] Instance Adaptive Self-Training for Unsupervised Domain Adaptation

Ke Mei, Chuang Zhu+2 more|arXiv (Cornell University)|Aug 27, 2020

Domain Adaptation and Few-Shot Learning35 references19 citations

TL;DR

This paper proposes Instance Adaptive Self-Training (IAST), a novel self-training framework for unsupervised domain adaptation in semantic segmentation. It introduces an instance adaptive selector to improve pseudo-label quality by dynamically adjusting selection based on instance-level confidence and a region-guided regularization to refine predictions, achieving state-of-the-art performance on GTA5-to-Cityscapes (52.2% mIoU) and SYNTHIA-to-Cityscapes (57.0% mIoU*) with high scalability across methods.

ABSTRACT

The divergence between labeled training data and unlabeled testing data is a significant challenge for recent deep learning models. Unsupervised domain adaptation (UDA) attempts to solve such a problem. Recent works show that self-training is a powerful approach to UDA. However, existing methods have difficulty in balancing scalability and performance. In this paper, we propose an instance adaptive self-training framework for UDA on the task of semantic segmentation. To effectively improve the quality of pseudo-labels, we develop a novel pseudo-label generation strategy with an instance adaptive selector. Besides, we propose the region-guided regularization to smooth the pseudo-label region and sharpen the non-pseudo-label region. Our method is so concise and efficient that it is easy to be generalized to other unsupervised domain adaptation methods. Experiments on 'GTA5 to Cityscapes' and 'SYNTHIA to Cityscapes' demonstrate the superior performance of our approach compared with the state-of-the-art methods.

Motivation & Objective

• To address the challenge of domain shift in semantic segmentation by improving pseudo-label quality in unsupervised domain adaptation (UDA).
• To overcome limitations in existing self-training methods that suffer from redundant, low-quality pseudo-labels due to fixed confidence thresholds.
• To develop a scalable, modular framework that enhances performance without requiring model-specific architectural changes.
• To enable effective adaptation to both unsupervised and semi-supervised learning settings through a generalizable training strategy.

Proposed method

• Introduces an instance adaptive selector that adjusts pseudo-label generation based on per-instance confidence, increasing diversity and reducing redundancy.
• Employs a region-guided regularization that smooths predictions within pseudo-labeled regions and sharpens them outside, improving spatial consistency.
• Uses a two-phase training pipeline: a warm-up phase with a non-self-training method (e.g., adversarial training), followed by self-training with the proposed components.
• Applies the framework to existing UDA methods like AdaptSeg and AdvEnt, demonstrating plug-and-play compatibility and performance gains.
• Utilizes a rank-based confidence strategy per class across images to guide the adaptive selector, avoiding over-reliance on high-confidence pixels.
• Maintains simplicity and efficiency by avoiding complex module coupling, enabling broad applicability to other UDA and semi-supervised learning tasks.

Experimental results

Research questions

• RQ1 Can an instance-adaptive pseudo-label generation strategy improve the quality and diversity of pseudo-labels in self-training for UDA?
• RQ2 Does region-guided regularization enhance spatial consistency and prediction accuracy in the absence of ground truth in the target domain?
• RQ3 Can the proposed IAST framework be effectively applied to non-self-training UDA methods without architectural modifications?
• RQ4 How does IAST perform compared to state-of-the-art methods on challenging domain shift benchmarks like GTA5-to-Cityscapes and SYNTHIA-to-Cityscapes?
• RQ5 Can IAST be extended to semi-supervised learning with limited labeled data in the target domain?

Key findings

• IAST achieves 52.2% mIoU on the GTA5-to-Cityscapes benchmark, outperforming the previous SOTA method BLF by 1.4 percentage points.
• On the more challenging SYNTHIA-to-Cityscapes benchmark, IAST achieves 57.0% mIoU*, significantly exceeding the next best method by 3.2 percentage points.
• When applied to non-self-training UDA methods, IAST improves AdaptSeg [27] by +7.8% mIoU and AdvEnt [29] by +4.4% mIoU on GTA5-to-Cityscapes.
• IAST achieves 69.8% mIoU with only 1/2 of the labeled data on Cityscapes, outperforming prior semi-supervised methods under the same setting.
• The method demonstrates strong generalization, improving performance on both high- and low-domain-gap benchmarks without architectural dependencies.
• Ablation studies confirm that both the instance adaptive selector and region-guided regularization contribute significantly to performance gains.