[Paper Review] Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation

Qiming Zhang, Jing Zhang+2 more|arXiv (Cornell University)|Oct 28, 2019

Domain Adaptation and Few-Shot Learning46 references114 citations

TL;DR

The paper introduces a category anchor-guided UDA model (CAG-UDA) for semantic segmentation that uses category-wise anchors to identify active target samples, assign pseudo-labels, and align features across domains, achieving state-of-the-art results on GTA5→Cityscapes and SYNTHIA→Cityscapes.

ABSTRACT

Unsupervised domain adaptation (UDA) aims to enhance the generalization capability of a certain model from a source domain to a target domain. UDA is of particular significance since no extra effort is devoted to annotating target domain samples. However, the different data distributions in the two domains, or \\emph{domain shift/discrepancy}, inevitably compromise the UDA performance. Although there has been a progress in matching the marginal distributions between two domains, the classifier favors the source domain features and makes incorrect predictions on the target domain due to category-agnostic feature alignment. In this paper, we propose a novel category anchor-guided (CAG) UDA model for semantic segmentation, which explicitly enforces category-aware feature alignment to learn shared discriminative features and classifiers simultaneously. First, the category-wise centroids of the source domain features are used as guided anchors to identify the active features in the target domain and also assign them pseudo-labels. Then, we leverage an anchor-based pixel-level distance loss and a discriminative loss to drive the intra-category features closer and the inter-category features further apart, respectively. Finally, we devise a stagewise training mechanism to reduce the error accumulation and adapt the proposed model progressively. Experiments on both the GTA5$\ ightarrow $Cityscapes and SYNTHIA$\ ightarrow $Cityscapes scenarios demonstrate the superiority of our CAG-UDA model over the state-of-the-art methods. The code is available at \\url{https://github.com/RogerZhangzz/CAG_UDA}.

Motivation & Objective

• Address domain shift in unsupervised domain adaptation for semantic segmentation.
• Propose category anchors to enable explicit category-wise feature alignment and reliable pseudo-labels.
• Develop a stagewise training mechanism to reduce error accumulation from pseudo-labels.
• Demonstrate state-of-the-art performance on GTA5→Cityscapes and SYNTHIA→Cityscapes datasets.

Proposed method

• Compute category-wise centroids (category anchors) from source domain features.
• Identify active target samples by measuring distance to anchors and assign pseudo-labels.
• Use an anchor-based pixel-level distance loss to reduce intra-category variance.
• Use a discriminative loss to increase inter-category variance and refine decision boundaries.
• Implement a stagewise training procedure to stabilize pseudo-labels and progressive adaptation.

Experimental results

Research questions

• RQ1 Can category anchors guide effective unsupervised domain adaptation for semantic segmentation?
• RQ2 Do anchor-based alignment and pseudo-labeling improve target-domain segmentation performance compared to label-free alignment methods?
• RQ3 Does stagewise training mitigate error accumulation from pseudo-labels during UDA?
• RQ4 How does CAG-UDA perform on GTA5→Cityscapes and SYNTHIA→Cityscapes relative to state-of-the-art methods?

Key findings

• CAG-UDA achieves a mean IoU (mIoU) of 50.2 on GTA5→Cityscapes in Stage 3, surpassing prior SOTA methods.
• On GTA5→Cityscapes testing, CAG-UDA attains 51.7 mIoU across 19 classes.
• In SYNTHIA→Cityscapes, CAG-UDA outperforms previous methods for both 13-category (mIoU*) and 16-category (mIoU) benchmarks.
• Stagewise training with category anchors yields substantial gains over a single-stage approach and warm-up only.
• Category anchor-based pseudo-labels provide more reliable supervision than probability-based pseudo-labels in ablations.