[Paper Review] Object Detection as a Positive-Unlabeled Problem

Yuewei Yang, Kevin J Liang+1 more|arXiv (Cornell University)|Feb 11, 2020

Advanced Neural Network Applications18 references17 citations

TL;DR

This paper proposes treating object detection as a positive-unlabeled (PU) learning problem to address the issue of incomplete annotations in real-world datasets, where missing object bounding boxes lead to misleading training signals under standard positive-negative (PN) cross-entropy loss. By reformulating classification to avoid assuming unlabeled regions are negative, the authors introduce a PU loss that dynamically estimates positive class priors, achieving consistent performance gains across PASCAL VOC, MS COCO, Visual Genome, and DeepLesion under varying label missingness.

ABSTRACT

As with other deep learning methods, label quality is important for learning modern convolutional object detectors. However, the potentially large number and wide diversity of object instances that can be found in complex image scenes makes constituting complete annotations a challenging task; objects missing annotations can be observed in a variety of popular object detection datasets. These missing annotations can be problematic, as the standard cross-entropy loss employed to train object detection models treats classification as a positive-negative (PN) problem: unlabeled regions are implicitly assumed to be background. As such, any object missing a bounding box results in a confusing learning signal, the effects of which we observe empirically. To remedy this, we propose treating object detection as a positive-unlabeled (PU) problem, which removes the assumption that unlabeled regions must be negative. We demonstrate that our proposed PU classification loss outperforms the standard PN loss on PASCAL VOC and MS COCO across a range of label missingness, as well as on Visual Genome and DeepLesion with full labels.

Motivation & Objective

• To address the problem of incomplete object annotations in real-world datasets, which leads to misleading learning signals when using standard positive-negative (PN) classification.
• To recognize that missing annotations do not imply absence of objects, challenging the PN assumption that unlabeled regions are background.
• To propose a positive-unlabeled (PU) learning framework for object detection that better models the true data distribution in partially labeled datasets.
• To demonstrate that PU learning improves detection performance across diverse datasets with varying degrees of label missingness.
• To extend the applicability of PU learning beyond binary classification to multi-class detection and potential use in segmentation and action detection.

Proposed method

• Reformulate object detection as a PU learning problem by removing the assumption that unlabeled regions are negative, instead treating them as potentially positive or unknown.
• Introduce a PU classification loss that dynamically estimates the prior probability of the positive class using a running average of positive predictions during training.
• Modify the standard cross-entropy loss in Faster R-CNN’s Region Proposal Network (RPN) to use the PU objective, preserving the model architecture with minimal changes.
• Estimate the positive class prior π using a moving average of the model’s predictions on positive samples, enabling end-to-end training without additional supervision.
• Apply the PU loss only to the binary objectness classifier in the RPN, while keeping the rest of the detection head unchanged.
• Extend the approach to multi-class detection by estimating multiple class priors π_c for each object category, though this is left for future work.

Experimental results

Research questions

• RQ1 Can object detection performance be improved by treating incomplete annotations as a PU learning problem rather than a PN problem?
• RQ2 How does the PU loss perform under varying degrees of label missingness across diverse datasets?
• RQ3 Does the PU objective outperform standard PN cross-entropy loss in real-world datasets with known annotation incompleteness?
• RQ4 Can the PU approach generalize to medical imaging datasets where labeling is expensive and inconsistent?
• RQ5 What is the impact of dynamically estimating the positive class prior on model convergence and detection accuracy?

Key findings

• The PU loss outperforms standard PN cross-entropy loss on PASCAL VOC and MS COCO across all tested levels of label missingness, with consistent gains in mean average precision (mAP).
• On Visual Genome and DeepLesion—datasets with high annotation difficulty and expert-limited labeling—the PU objective achieves significant performance improvements over PN baselines.
• In DeepLesion, switching from PN to PU objective improves lesion detection sensitivity and FROC scores across multiple false positive allowances per image.
• The proposed PU loss achieves better performance even when the model is trained on datasets with up to 30% missing annotations, demonstrating robustness to label incompleteness.
• The dynamic estimation of the positive class prior via a running average of predictions enables effective PU learning with minimal architectural changes.
• The gains from PU learning are consistent across multiple datasets, suggesting that the PN assumption in standard object detectors is suboptimal when labels are incomplete.