[Paper Review] Collective Loss Function for Positive and Unlabeled Learning

Chenhao Xie, Qiao Cheng+3 more|arXiv (Cornell University)|May 6, 2020

Machine Learning and Data Classification31 references4 citations

TL;DR

This paper proposes cPU, a novel collective loss function for positive and unlabeled (PU) learning that rectifies predictions before loss calculation to address bias from unlabeled data uncertainty. By theoretically eliciting a loss function through predictor-level collective correction, cPU achieves state-of-the-art performance and robustness on benchmark and real-world datasets, outperforming existing methods by 1–4 accuracy points on CIFAR-10 and showing stable results under prior estimation errors.

ABSTRACT

People learn to discriminate between classes without explicit exposure to negative examples. On the contrary, traditional machine learning algorithms often rely on negative examples, otherwise the model would be prone to collapse and always-true predictions. Therefore, it is crucial to design the learning objective which leads the model to converge and to perform predictions unbiasedly without explicit negative signals. In this paper, we propose a Collectively loss function to learn from only Positive and Unlabeled data (cPU). We theoretically elicit the loss function from the setting of PU learning. We perform intensive experiments on the benchmark and real-world datasets. The results show that cPU consistently outperforms the current state-of-the-art PU learning methods.

Motivation & Objective

• To address the challenge of biased risk estimation in PU learning due to class uncertainty in unlabeled data.
• To develop a method that minimally intervenes in feature engineering while maintaining high model flexibility.
• To design a robust, theoretically grounded loss function that corrects decision boundary drift without relying on explicit negative examples.
• To improve generalization and performance in PU learning by rectifying predictors collectively before loss computation.

Proposed method

• The method introduces a collective prediction mechanism that aggregates outputs from multiple predictors to reduce noise and uncertainty from unlabeled data.
• It derives a new loss function through theoretical elicitation, focusing on correcting the predictor rather than the risk estimator, to eliminate bias from class uncertainty.
• The approach uses a sigmoid-activated neural network classifier with a modified loss function that accounts for posterior probability estimation under PU constraints.
• The collective loss function is designed to be Fisher consistent and bounded in generalization error, ensuring convergence to optimal decision boundaries.
• It avoids explicit negative labeling by treating unlabeled data as a mixture of positive and negative samples and using collective inference to balance predictions.
• The method is applied end-to-end with deep neural networks, preserving model capacity while improving robustness to inaccurate class prior estimates.

Experimental results

Research questions

• RQ1 How can we design a loss function for PU learning that avoids bias from uncertain class labels in unlabeled data?
• RQ2 Can collective prediction correction before loss computation lead to better generalization and performance than risk-based correction?
• RQ3 How robust is the proposed method to inaccurately estimated class priors in the unlabeled set?
• RQ4 Does the collective loss function outperform existing state-of-the-art PU learning methods across diverse benchmark and real-world datasets?

Key findings

• cPU consistently outperforms state-of-the-art PU learning methods, achieving a 1–4 point accuracy gain on the challenging CIFAR-10 dataset.
• On MNIST, USPS, SVHN, and 20ng datasets, cPU maintains high performance with minimal standard deviation, indicating strong stability.
• The method shows robustness to misestimated class priors: performance fluctuation remains acceptable even when the positive prior in unlabeled data varies by ±10%.
• Training visualization shows that unlabeled positive samples (e.g., dogs) gradually move toward the positive cluster, while ambiguous or noisy samples oscillate near the decision boundary.
• Case studies reveal that model predictions are guided by label signals and visual similarity, with mispredictions occurring mainly on ambiguous or human-containing images.
• Theoretical analysis confirms that the collective loss function is Fisher consistent and generalization error is bounded, supporting convergence to optimal decision boundaries.