[Paper Review] MixPUL: Consistency-based Augmentation for Positive and Unlabeled Learning

Wei Tong, Feng Shi+2 more|arXiv (Cornell University)|Apr 20, 2020

Machine Learning and Data Classification24 references4 citations

TL;DR

MixPUL proposes a consistency-based data augmentation method for positive and unlabeled (PU) learning that leverages mixup and supervised/unsupervised consistency regularization without requiring class-prior knowledge. It improves AUC and reduces classification error by mining reliable negative examples from unlabeled data and applying margin loss, achieving a 20.6% relative reduction in error (from 16.49 to 13.09) on CIFAR-10 across varying positive set sizes.

ABSTRACT

Learning from positive and unlabeled data (PU learning) is prevalent in practical applications where only a couple of examples are positively labeled. Previous PU learning studies typically rely on existing samples such that the data distribution is not extensively explored. In this work, we propose a simple yet effective data augmentation method, coined~\algo, based on \emph{consistency regularization} which provides a new perspective of using PU data. In particular, the proposed~\algo~incorporates supervised and unsupervised consistency training to generate augmented data. To facilitate supervised consistency, reliable negative examples are mined from unlabeled data due to the absence of negative samples. Unsupervised consistency is further encouraged between unlabeled datapoints. In addition,~\algo~reduces margin loss between positive and unlabeled pairs, which explicitly optimizes AUC and yields faster convergence. Finally, we conduct a series of studies to demonstrate the effectiveness of consistency regularization. We examined three kinds of reliable negative mining methods. We show that~\algo~achieves an averaged improvement of classification error from 16.49 to 13.09 on the CIFAR-10 dataset across different positive data amount.

Motivation & Objective

• To address the limitation of existing PU learning methods that rely on class-prior estimates, which are often inaccurate and degrade performance.
• To improve generalization and AUC in PU learning by introducing consistency regularization using mixup on both labeled and unlabeled data.
• To develop a robust, prior-free method that effectively utilizes unlabeled data without assuming known data distributions.
• To evaluate the effectiveness of different reliable negative mining strategies in the absence of true negative labels.
• To demonstrate that consistency regularization and margin loss jointly accelerate convergence and improve model stability.

Proposed method

• MixPUL applies mixup-based consistency regularization by interpolating pairs of data points (positive and unlabeled) and their labels, encouraging the model to predict interpolated labels consistently.
• It introduces a supervised consistency loss by mining reliable negative examples from unlabeled data to simulate negative supervision, enabling better generalization.
• An unsupervised consistency loss is enforced between unlabeled data points via mixup, promoting robustness in low-density regions of the data manifold.
• A margin loss is introduced between positive and unlabeled pairs to explicitly minimize the risk of AUC, encouraging higher model confidence on positive examples.
• The method combines mixup, negative mining, and margin loss into a unified objective function that jointly optimizes for AUC and consistency.
• Three negative mining techniques are evaluated: distance-based (Dist), NTC (random forest), and randomized downsampling (Rand), with Rand showing superior performance.

Experimental results

Research questions

• RQ1 Can consistency-based data augmentation via mixup improve performance in PU learning without requiring class-prior estimates?
• RQ2 How effective are different reliable negative mining strategies in the absence of true negative labels?
• RQ3 Does unsupervised mixup on unlabeled data improve model generalization and convergence speed in PU learning?
• RQ4 To what extent does incorporating margin loss between positive and unlabeled pairs enhance AUC and model stability?
• RQ5 How does the combination of mixup, negative mining, and margin loss compare to state-of-the-art PU learning methods like uPU and nnPU?

Key findings

• MixPUL achieves an averaged classification error reduction from 16.49 to 13.09 on CIFAR-10 across different positive set sizes, representing a 20.6% relative improvement.
• The randomized negative mining method outperforms distance-based and NTC-based approaches, especially in low-data regimes.
• Unsupervised mixup on unlabeled data significantly improves generalization, reducing error to below 10% after initial training instability.
• Incorporating margin loss leads to faster convergence and more stable training, particularly when combined with mixup.
• MixPUL outperforms state-of-the-art methods uPU and nnPU even without access to true class-prior values, especially when positive data is scarce.
• Ablation studies confirm that all components—mixup, margin loss, and negative mining—are essential; removing any leads to performance degradation or model collapse.