[Paper Review] Known-class Aware Self-ensemble for Open Set Domain Adaptation

Qing Lian, Wen Li+2 more|arXiv (Cornell University)|May 3, 2019

Domain Adaptation and Few-Shot Learning25 references8 citations

TL;DR

This paper proposes Known-class Aware Self-ensemble (KASE), a novel method for open set domain adaptation that addresses partial label space overlap between source and target domains. KASE integrates a Known-class Aware Recognition (KAR) module to distinguish known from unknown classes and a Known-class Aware Adaptation (KAA) module to reweight adaptation loss based on known-class likelihood, significantly improving performance over state-of-the-art methods on Syn2Real-O, Office-31, and Digits datasets.

ABSTRACT

Existing domain adaptation methods generally assume different domains have the identical label space, which is quite restrict for real-world applications. In this paper, we focus on a more realistic and challenging case of open set domain adaptation. Particularly, in open set domain adaptation, we allow the classes from the source and target domains to be partially overlapped. In this case, the assumption of conventional distribution alignment does not hold anymore, due to the different label spaces in two domains. To tackle this challenge, we propose a new approach coined as Known-class Aware Self-Ensemble (KASE), which is built upon the recently developed self-ensemble model. In KASE, we first introduce a Known-class Aware Recognition (KAR) module to identify the known and unknown classes from the target domain, which is achieved by encouraging a low cross-entropy for known classes and a high entropy based on the source data from the unknown class. Then, we develop a Known-class Aware Adaptation (KAA) module to better adapt from the source domain to the target by reweighing the adaptation loss based on the likeliness to belong to known classes of unlabeled target samples as predicted by KAR. Extensive experiments on multiple benchmark datasets demonstrate the effectiveness of our approach.

Motivation & Objective

• To address the challenge of open set domain adaptation where source and target domains share only a subset of classes.
• To overcome the failure of conventional domain adaptation methods that assume identical label spaces.
• To improve model robustness by distinguishing known from unknown classes in the target domain.
• To enhance domain adaptation by reweighting unlabeled target samples based on their likelihood of belonging to known classes.
• To achieve better performance in realistic scenarios with partial class overlap and unknown classes.

Proposed method

• KARE uses a dual loss: minimizing cross-entropy for known classes and maximizing entropy for unknown classes in the source domain.
• The Known-class Aware Recognition (KAR) module classifies target samples as known or unknown based on predicted confidence and entropy.
• The Known-class Aware Adaptation (KAA) module reweights the adaptation loss using continuous scores from KAR to prioritize known-class samples.
• KASE is built on the self-ensemble framework, enabling end-to-end training of KAR and KAA modules.
• The method leverages feature representations from a shared backbone (e.g., AlexNet) and applies t-SNE visualization to analyze feature distribution shifts.
• A continuous reweighting strategy replaces binary weighting, reducing negative impact from unknown-class target samples.

Experimental results

Research questions

• RQ1 How can a model effectively distinguish known from unknown classes in the target domain under open set domain adaptation?
• RQ2 Can reweighting adaptation loss based on known-class likelihood improve domain adaptation performance?
• RQ3 How does the proposed KASE method handle domain shift when label spaces are partially overlapping?
• RQ4 Does the integration of KAR and KAA modules outperform existing open set and closed set domain adaptation methods?
• RQ5 To what extent does the continuous reweighting strategy mitigate negative transfer from unknown-class samples?

Key findings

• On the Syn2Real-O dataset, KASE achieved 87.0% accuracy on the A→R setting, outperforming the previous SOTA of 86.9%.
• On Office-31, KASE achieved the highest mean accuracy of 81.8% across all six domain transfer settings, surpassing prior methods.
• On the Digits dataset, KASE achieved 84.8% mean accuracy, exceeding the SOTA of 82.8% reported by AODA.
• Visualization via t-SNE confirmed that known-class features were well-separated and clustered, while unknown-class samples showed higher entropy and dispersion.
• The ablation study showed that KAR’s entropy maximization for unknown classes significantly improved generalization and reduced misclassification.
• The continuous reweighting strategy in KAA led to more stable and effective adaptation, especially in high-shift settings like D→A and W→A.