[Paper Review] Relaxed Conditional Image Transfer for Semi-supervised Domain Adaptation

Qijun Luo, Zhili Liu+8 more|arXiv (Cornell University)|Jan 5, 2021

Domain Adaptation and Few-Shot Learning9 references4 citations

TL;DR

This paper proposes Relaxed cGAN, a novel conditional GAN framework for semi-supervised domain adaptation that alleviates the label-domination problem by removing class labels from the generator input, forcing it to infer semantics from raw images. The method achieves state-of-the-art performance on DomainNet, Office-Home, and multiple digit benchmarks under both low- and high-resource settings.

ABSTRACT

Semi-supervised domain adaptation (SSDA), which aims to learn models in a partially labeled target domain with the assistance of the fully labeled source domain, attracts increasing attention in recent years. To explicitly leverage the labeled data in both domains, we naturally introduce a conditional GAN framework to transfer images without changing the semantics in SSDA. However, we identify a label-domination problem in such an approach. In fact, the generator tends to overlook the input source image and only memorizes prototypes of each class, which results in unsatisfactory adaptation performance. To this end, we propose a simple yet effective Relaxed conditional GAN (Relaxed cGAN) framework. Specifically, we feed the image without its label to our generator. In this way, the generator has to infer the semantic information of input data. We formally prove that its equilibrium is desirable and empirically validate its practical convergence and effectiveness in image transfer. Additionally, we propose several techniques to make use of unlabeled data in the target domain, enhancing the model in SSDA settings. We validate our method on the well-adopted datasets: Digits, DomainNet, and Office-Home. We achieve state-of-the-art performance on DomainNet, Office-Home and most digit benchmarks in low-resource and high-resource settings.

Motivation & Objective

• To address the label-domination problem in conditional GAN-based image transfer for semi-supervised domain adaptation, where generators memorize class prototypes instead of using source image features.
• To improve image transfer quality and semantic consistency by training the generator with only input images, not labels, while the discriminator enforces class-conditional alignment.
• To enhance model generalization in low-resource settings by leveraging unlabeled target data through marginal and pseudo-label losses.
• To provide theoretical justification for the equilibrium of the proposed Relaxed cGAN framework, ensuring stable training and convergence.

Proposed method

• The generator takes only the source image as input, without class labels, to force it to learn semantic features directly from the image content.
• The discriminator receives both the generated image and its true label, enabling it to distinguish real from fake image-label pairs in a class-conditional manner.
• The framework uses a conditional GAN loss with a relaxed condition, where the generator is not conditioned on labels, reducing label dominance.
• A marginal loss is introduced to align the marginal distribution of generated images with that of real target images, improving domain alignment.
• A pseudo-labeling loss is applied to unlabeled target samples, using the model’s own predictions to refine feature learning.
• Theoretical analysis proves that the equilibrium of the proposed Relaxed cGAN is desirable, ensuring stable training dynamics.

Experimental results

Research questions

• RQ1 Does removing label conditioning from the generator mitigate the label-domination problem in conditional GANs for semi-supervised domain adaptation?
• RQ2 Can a generator trained without labels still produce high-quality, semantically consistent image translations in the target domain?
• RQ3 Does the proposed Relaxed cGAN framework achieve better performance than existing feature-level and pixel-level adaptation methods on standard benchmarks?
• RQ4 How effective are the additional regularizations (marginal and pseudo-loss) in improving model generalization under low-resource settings?

Key findings

• Relaxed cGAN achieves state-of-the-art performance on the DomainNet dataset under the 3-shot setting, outperforming all baselines with an average accuracy of 74.2%.
• On the Office-Home benchmark, Relaxed cGAN achieves the highest average accuracy of 98.7% in the 1-shot setting, surpassing all compared methods.
• In ablation studies, Relaxed cGAN without additional losses achieves 94.3% accuracy on the S→M task, significantly outperforming cycleGAN (68.4%) and the preliminary method (87.8%).
• The addition of marginal and pseudo-label losses further improves performance, increasing accuracy from 94.3% to 95.9% on the S→M task.
• Empirical results confirm that the proposed method effectively resolves the label-inconsistency and label-domination problems observed in standard conditional GANs.
• Theoretical analysis confirms that the equilibrium of the Relaxed cGAN framework is stable and desirable, supporting practical convergence.