[Paper Review] Mining Multi-Label Samples from Single Positive Labels

Youngin Cho, Daejin Kim+2 more|arXiv (Cornell University)|Jun 12, 2022

Generative Adversarial Networks and Image Synthesis4 citations

TL;DR

This paper proposes S2M sampling, a novel Markov chain Monte Carlo-based method that enables conditional GANs to generate high-quality multi-label images from single positive labels—where only one label is provided per sample—without requiring costly multi-label annotations. The approach achieves performance comparable to models trained on fully annotated datasets, even outperforming them on downstream classification and FID metrics.

ABSTRACT

Conditional generative adversarial networks (cGANs) have shown superior results in class-conditional generation tasks. To simultaneously control multiple conditions, cGANs require multi-label training datasets, where multiple labels can be assigned to each data instance. Nevertheless, the tremendous annotation cost limits the accessibility of multi-label datasets in real-world scenarios. Therefore, in this study we explore the practical setting called the single positive setting, where each data instance is annotated by only one positive label with no explicit negative labels. To generate multi-label data in the single positive setting, we propose a novel sampling approach called single-to-multi-label (S2M) sampling, based on the Markov chain Monte Carlo method. As a widely applicable "add-on" method, our proposed S2M sampling method enables existing unconditional and conditional GANs to draw high-quality multi-label data with a minimal annotation cost. Extensive experiments on real image datasets verify the effectiveness and correctness of our method, even when compared to a model trained with fully annotated datasets.

Motivation & Objective

• Address the high annotation cost of creating multi-label datasets for conditional image generation.
• Enable generation of multi-label data when only single positive labels are available per sample.
• Preserve image quality and sample diversity while generating from overlapping and non-overlapping joint classes.
• Develop a plug-and-play sampling framework compatible with existing unconditional and conditional GANs.
• Model intrinsic class relationships without requiring explicit negative labels or complex model retraining.

Proposed method

• Propose a single-to-multi-label (S2M) sampling framework that uses Markov chain Monte Carlo (MCMC) to sample from joint class distributions.
• Estimate conditional densities of joint classes using multiple classification networks to approximate the target distribution.
• Apply the Independent Metropolis-Hastings algorithm to generate samples from the joint class distribution by leveraging the generator’s output and classifier scores.
• Use latent space adaptation to improve sample efficiency and convergence in high-dimensional data space.
• Integrate the sampling process at inference time, preserving the original GAN’s architecture and objective.
• Formulate a tractable density ratio estimation method to compute $ p_t(x)/p_G(x) $ for MCMC acceptance decisions.

Experimental results

Research questions

• RQ1 Can multi-label samples be effectively generated from a single positive label setting without explicit negative labels?
• RQ2 How can overlapping and non-overlapping joint classes be accurately sampled using only one positive label per instance?
• RQ3 Can a post-processing sampling method achieve performance comparable to models trained on fully annotated multi-label datasets?
• RQ4 Does the proposed S2M sampling framework maintain high image quality while enabling diverse, attribute-combined generation?
• RQ5 How does the method scale across different datasets and attribute combinations with minimal hyperparameter tuning?

Key findings

• S2M sampling achieved 74.97% accuracy on CelebA-BBM and 70.94% on CelebA-HBW, significantly outperforming baseline StyleGANv2 (18.41% and 12.68%) when generating joint-class samples.
• FID scores improved from 15.57 to 14.41 on CelebA-BBM and from 15.80 to 14.54 on CelebA-HBW, indicating higher sample quality.
• Qualitative results show diverse, realistic samples across all joint classes, including complex combinations like 'brown hair, bushy eyebrows, mouth slightly open'.
• The method successfully generates samples from both overlapping (e.g., A ∩ B) and non-overlapping (e.g., A \\( B) classes using only single positive labels.
• S2M sampling is effective across diverse datasets (MNIST, CIFAR-10, CelebA) and works as a plug-in solution with existing GANs without architectural or objective changes.
• The Independent Metropolis-Hastings variant outperformed rejection sampling and Langevin dynamics in stability and sample quality, especially in high-dimensional spaces.