[Paper Review] Adversarial Style Mining for One-Shot Unsupervised Domain Adaptation

Yawei Luo, Ping Liu+3 more|arXiv (Cornell University)|Apr 13, 2020

Domain Adaptation and Few-Shot Learning45 references66 citations

TL;DR

The paper introduces Adversarial Style Mining (ASM), an end-to-end framework that combines a Random AdaIN style generator with a task model to perform one-shot unsupervised domain adaptation by iteratively mining harder target-like styles guided by feedback from the task model.

ABSTRACT

We aim at the problem named One-Shot Unsupervised Domain Adaptation. Unlike traditional Unsupervised Domain Adaptation, it assumes that only one unlabeled target sample can be available when learning to adapt. This setting is realistic but more challenging, in which conventional adaptation approaches are prone to failure due to the scarce of unlabeled target data. To this end, we propose a novel Adversarial Style Mining approach, which combines the style transfer module and task-specific module into an adversarial manner. Specifically, the style transfer module iteratively searches for harder stylized images around the one-shot target sample according to the current learning state, leading the task model to explore the potential styles that are difficult to solve in the almost unseen target domain, thus boosting the adaptation performance in a data-scarce scenario. The adversarial learning framework makes the style transfer module and task-specific module benefit each other during the competition. Extensive experiments on both cross-domain classification and segmentation benchmarks verify that ASM achieves state-of-the-art adaptation performance under the challenging one-shot setting.

Motivation & Objective

• Address the one-shot unsupervised domain adaptation (OSUDA) problem where only a single unlabeled target sample is available during training.
• Enable the model to explore target-domain styles beyond the one-shot sample to improve generalization.
• Propose a differentiable, adversarial framework that jointly optimizes style generation and task performance.
• Introduce a novel Random AdaIN (RAIN) module to enable differentiable style searching within ASM.
• Demonstrate state-of-the-art performance on cross-domain classification and semantic segmentation under one-shot settings.

Proposed method

• We form ASM as an end-to-end adversarial loop between a stylized image generator G and a task-specific model M, where G is driven by an anchor target style and augmented by a Random AdaIN (RAIN) module.
• RAIN extends AdaIN with a style VAE to encode and sample styles, making style generation differentiable and gradient-friendly for end-to-end learning.
• A sampling vector ε, drawn from the style distribution learned from the one-shot target, is perturbed iteratively based on the gradient of the task loss to mine harder styles.
• The loss for training M combines a task loss on stylized samples with a consistency loss to encourage semantic stability across different stylizations, forming L_M = L_task + λ L_consist.
• The style mining procedure iteratively updates ε for a predefined depth n to produce progressively harder stylized samples around the anchor style.
• Training uses source data with labels and the single target sample as the style anchor, without requiring target labels.

Experimental results

Research questions

• RQ1 Can one-shot unlabeled target data be leveraged to learn robust domain-adaptive models via adversarial style mining?
• RQ2 Does a differentiable style generation mechanism (RAIN) enable effective end-to-end OSUDA by continuously providing harder yet relevant target-style variations?
• RQ3 How does ASM compare to existing OSUDA and one-shot style-transfer methods on classification and segmentation benchmarks?
• RQ4 What are the stability properties and style-distribution characteristics of the mined styles during training?

Key findings

• ASM achieves state-of-the-art results on one-shot cross-domain classification benchmarks MNIST-USPS-SVHN, outperforming several UDA and one-shot style-transfer baselines.
• In classification tasks, ASM attains 46.3% on M→S and 40.3% on U→S, exceeding competing methods under the one-shot setting.
• In segmentation benchmarks (SYNTHIA/GTA5 and Cityscapes), ASM yields superior mean IoU compared with several DA methods under one-shot and approaches or surpasses some few-shot baselines.
• The method maintains competitive performance under conventional (data-rich) UDA and demonstrates a smaller performance drop under one-shot constraints, indicating robust style mining.
• Training analysis shows that the adversarial loop between M and the style vector ε is stable, with L_M converging to a small value and mined styles lying around the anchor target style.
• Empirical visualizations (t-SNE) suggest mined styles cover unseen yet related target styles around the anchor style.