[Paper Review] Generative Adversarial Positive-Unlabelled Learning

Ming Hou, Brahim Chaib-draa+2 more|arXiv (Cornell University)|Nov 21, 2017

Machine Learning and Data Classification4 citations

TL;DR

This paper proposes GenPU, a novel generative adversarial framework for positive-unlabeled (PU) learning that uses separate generators and discriminators to synthesize realistic positive and negative samples, enabling robust training of deep neural networks even with very limited labeled positive data. At equilibrium, GenPU theoretically recovers both data distributions and achieves state-of-the-art accuracy, especially under low-data regimes, outperforming existing PU methods in overfitting resilience and generalization.

ABSTRACT

In this work, we consider the task of classifying binary positive-unlabeled (PU) data. The existing discriminative learning based PU models attempt to seek an optimal reweighting strategy for U data, so that a decent decision boundary can be found. However, given limited P data, the conventional PU models tend to suffer from overfitting when adapted to very flexible deep neural networks. In contrast, we are the first to innovate a totally new paradigm to attack the binary PU task, from perspective of generative learning by leveraging the powerful generative adversarial networks (GAN). Our generative positive-unlabeled (GenPU) framework incorporates an array of discriminators and generators that are endowed with different roles in simultaneously producing positive and negative realistic samples. We provide theoretical analysis to justify that, at equilibrium, GenPU is capable of recovering both positive and negative data distributions. Moreover, we show GenPU is generalizable and closely related to the semi-supervised classification. Given rather limited P data, experiments on both synthetic and real-world dataset demonstrate the effectiveness of our proposed framework. With infinite realistic and diverse sample streams generated from GenPU, a very flexible classifier can then be trained using deep neural networks.

Motivation & Objective

• To address the overfitting problem in PU learning when using flexible deep neural networks with limited positive-labeled data.
• To develop a new paradigm for PU learning by leveraging generative modeling instead of discriminative reweighting strategies.
• To theoretically justify that the proposed framework can recover both positive and negative data distributions at equilibrium.
• To demonstrate generalization and robustness of the framework across diverse data distributions and low-data settings.

Proposed method

• The framework employs an array of discriminators and generators, each specialized to model positive or negative data distributions.
• Generators $G_p$ and $G_n$ synthesize realistic positive and negative samples from random noise, while discriminators $D_p$ and $D_n$ distinguish real from fake samples.
• The training process follows a minimax game where $G_p$ and $D_p$ jointly optimize to model the positive distribution, and $G_n$ and $D_n$ do the same for the negative distribution.
• The framework is generalizable and can be extended to other GAN variants using different divergence metrics, such as Wasserstein distance, to improve sample diversity and training stability.
• A standard positive-negative classifier is trained on the synthesized data streams, enabling high-capacity models without overfitting.
• Theoretical analysis proves that at equilibrium, the generators recover the true underlying data distributions of both positive and negative classes.

Experimental results

Research questions

• RQ1 Can a generative adversarial framework effectively model both positive and negative data distributions in PU learning?
• RQ2 Does the proposed GenPU framework mitigate overfitting in deep neural networks when only a few labeled positive samples are available?
• RQ3 How does the performance of GenPU compare to existing PU learning methods under low-data regimes?
• RQ4 To what extent is the GenPU framework generalizable across different GAN variants and divergence metrics?

Key findings

• GenPU achieves 98.3% accuracy on MNIST for the '3' vs '5' classification with only 100 labeled positive samples, outperforming UPU and NNPU.
• With just one labeled positive example, GenPU maintains 97.6% accuracy on the '3' vs '5' task, while NNPU drops to 56.3%.
• On the USPS dataset, GenPU achieves 94.0% accuracy with one labeled '3', significantly outperforming UPU (54.0%) and NNPU (61.0%).
• When the number of labeled positive samples is reduced to 5, GenPU maintains 97.9% accuracy on MNIST, while NNPU's accuracy drops to 84.3%.
• GenPU avoids overfitting even with highly flexible classifiers, as evidenced by stable training and test error curves across low-data settings.
• Theoretical analysis confirms that at equilibrium, GenPU recovers both the positive and negative data distributions, ensuring unbiased learning.