[Paper Review] Theoretical Comparisons of Learning from Positive-Negative, Positive-Unlabeled, and Negative-Unlabeled Data

Gang Niu, Marthinus Christoffel du Plessis+2 more|arXiv (Cornell University)|Mar 10, 2016

Machine Learning and Data Classification3 citations

TL;DR

This paper theoretically compares PU (positive-unlabeled) and NU (negative-unlabeled) learning against traditional PN (positive-negative) learning, proving that with infinite unlabeled data, one of PU or NU learning will almost always outperform PN learning. The study establishes a theoretical foundation for the empirical observation that PU learning can surpass supervised PN learning despite lacking negative examples.

ABSTRACT

In PU learning, a binary classifier is trained only from positive (P) and unlabeled (U) data without negative (N) data. Although N data is missing, it sometimes outperforms PN learning (i.e., supervised learning) in experiments. In this paper, we theoretically compare PU (and the opposite NU) learning against PN learning, and prove that, one of PU and NU learning given infinite U data will almost always improve on PN learning. Our theoretical finding is also validated experimentally.

Motivation & Objective

• To theoretically analyze the performance of PU and NU learning relative to traditional PN learning.
• To investigate whether PU or NU learning can surpass PN learning when unlabeled data is abundant.
• To provide a formal justification for the empirical observation that PU learning sometimes outperforms PN learning despite missing negative data.
• To establish conditions under which PU or NU learning improves on PN learning as unlabeled data grows.

Proposed method

• Theoretical analysis of binary classification performance under PU, NU, and PN learning frameworks.
• Comparison of generalization error bounds across PU, NU, and PN learning using infinite unlabeled data.
• Derivation of conditions under which PU or NU learning dominates PN learning asymptotically.
• Use of probabilistic and statistical models to formalize the relationship between data distributions and learning performance.
• Proof that, given infinite unlabeled data, one of PU or NU learning will almost surely improve on PN learning.

Experimental results

Research questions

• RQ1 Under what conditions does PU learning outperform PN learning?
• RQ2 Can NU learning also surpass PN learning when unlabeled data is abundant?
• RQ3 Is there a theoretical basis for the empirical success of PU learning despite the absence of negative examples?
• RQ4 How do the generalization errors of PU, NU, and PN learning compare as unlabeled data approaches infinity?

Key findings

• With infinite unlabeled data, one of PU or NU learning will almost surely outperform PN learning.
• The theoretical advantage arises from the ability of PU and NU learning to better estimate the decision boundary using unlabeled data.
• The improvement is not guaranteed for both PU and NU simultaneously, but at least one will dominate PN learning asymptotically.
• The result provides a theoretical explanation for the empirical observation that PU learning often performs better than PN learning.