[Paper Review] Progressive Graph Learning for Open-Set Domain Adaptation

Yadan Luo, Zijian Wang+2 more|arXiv (Cornell University)|Jun 22, 2020

Domain Adaptation and Few-Shot Learning39 citations

TL;DR

The paper introduces Progressive Graph Learning (PGL), an end-to-end open-set unsupervised domain adaptation method that uses episodic training, graph neural networks, and adversarial learning to handle unknown target classes and reduce conditional shift, achieving a tighter target error bound.

ABSTRACT

Domain shift is a fundamental problem in visual recognition which typically arises when the source and target data follow different distributions. The existing domain adaptation approaches which tackle this problem work in the closed-set setting with the assumption that the source and the target data share exactly the same classes of objects. In this paper, we tackle a more realistic problem of open-set domain shift where the target data contains additional classes that are not present in the source data. More specifically, we introduce an end-to-end Progressive Graph Learning (PGL) framework where a graph neural network with episodic training is integrated to suppress underlying conditional shift and adversarial learning is adopted to close the gap between the source and target distributions. Compared to the existing open-set adaptation approaches, our approach guarantees to achieve a tighter upper bound of the target error. Extensive experiments on three standard open-set benchmarks evidence that our approach significantly outperforms the state-of-the-arts in open-set domain adaptation.

Motivation & Objective

• Motivate open-set domain adaptation where target data contain unknown classes not seen in the source.
• Develop an end-to-end framework that minimizes source risk, domain discrepancy, and open-set risk under a progressive learning regime.
• Incorporate graph neural networks and episodic training to address conditional shift at sample and manifold levels.
• Provide theoretical bound insights showing tighter target error with the progressive approach.

Proposed method

• Decompose the hypothesis space into shared (known) and unknown subspaces to reduce misclassification of source data.
• Adopt progressive learning to gradually convert low-confidence target samples into pseudo-labeled knowns and unknowns to mitigate negative transfer.
• Use episodic training and a transductive setup to align conditional distributions by replacing source data with pseudo-labeled target data in episodes.
• Construct a graph neural network on top of backbone features, connecting same-class source and pseudo-labeled target nodes to learn class-specific manifolds.
• Incorporate an adversarial domain discriminator to minimize marginal distribution gap for known classes.
• Train with a joint objective that balances node classification loss, edge classification loss, and adversarial loss (with specified coefficients).
• Progressively update pseudo-labels and episodes across M steps, controlled by an enlarging factor alpha and openness parameter beta.

Experimental results

Research questions

• RQ1 How can open-set domain adaptation be theoretically bounded to account for unknown target classes?
• RQ2 Can a progressive, graph-based framework reduce conditional shift and domain discrepancy more effectively than prior OUDA methods?
• RQ3 Does episodic, transductive GNN-based learning improve shared-class representations and mitigate negative transfer?
• RQ4 What is the impact of pseudo-labeling known vs unknown samples on target risk and overall performance?

Key findings

• PGL consistently outperforms state-of-the-art open-set domain adaptation methods on Office-Home, Syn2Real-O, and VisDA-17 benchmarks.
• Progressive pseudo-labeling plus GNN-based manifold learning yields notable gains in OS* (known-class normalized accuracy) across datasets.
• Ablation studies show that removing progressive labeling, GNNs, or mix-up strategy degrades performance, confirming the importance of each component.
• The framework achieves higher robustness to openness and favorable sensitivity properties with respect to loss coefficients and enlarging factor alpha.
• Across datasets, PGL achieves substantial improvements in OS and OS* metrics compared to baselines like OSBP, STA, ATI-λ, MMD, and DANN.