[Paper Review] Mixed Supervised Object Detection with Robust Objectness Transfer

Yan Li, Junge Zhang+2 more|arXiv (Cornell University)|Feb 27, 2018

Advanced Neural Network Applications38 references5 citations

TL;DR

This paper proposes a robust objectness transfer method for mixed supervised object detection, leveraging domain-invariant objectness knowledge from fully labeled categories to improve weakly supervised detection of novel categories. By learning invariant features across domains and integrating them into a multiple instance learning framework, the approach significantly reduces false positives from distractors and achieves state-of-the-art performance on ILSVRC2013 and PASCAL VOC datasets.

ABSTRACT

In this paper, we consider the problem of leveraging existing fully labeled categories to improve the weakly supervised detection (WSD) of new object categories, which we refer to as mixed supervised detection (MSD). Different from previous MSD methods that directly transfer the pre-trained object detectors from existing categories to new categories, we propose a more reasonable and robust objectness transfer approach for MSD. In our framework, we first learn domain-invariant objectness knowledge from the existing fully labeled categories. The knowledge is modeled based on invariant features that are robust to the distribution discrepancy between the existing categories and new categories; therefore the resulting knowledge would generalize well to new categories and could assist detection models to reject distractors (e.g., object parts) in weakly labeled images of new categories. Under the guidance of learned objectness knowledge, we utilize multiple instance learning (MIL) to model the concepts of both objects and distractors and to further improve the ability of rejecting distractors in weakly labeled images. Our robust objectness transfer approach outperforms the existing MSD methods, and achieves state-of-the-art results on the challenging ILSVRC2013 detection dataset and the PASCAL VOC datasets.

Motivation & Objective

• To address the challenge of weakly supervised detection (WSD) where object detectors often confuse objects with co-occurring distractors such as object parts or contextual elements.
• To improve detection performance for novel object categories without bounding box annotations by leveraging existing fully labeled categories.
• To learn domain-invariant objectness knowledge from strong categories that generalizes well to unseen weak categories.
• To develop a method that robustly transfers objectness knowledge to reduce false positives in weakly labeled images.
• To outperform existing mixed supervised detection (MSD) methods by explicitly modeling both objects and distractors through objectness-aware multiple instance learning.

Proposed method

• The method learns domain-invariant objectness knowledge by training a feature extractor on fully labeled strong categories to minimize distribution discrepancy between strong and weak categories.
• Objectness is modeled using a deep neural network that learns to score regions based on completeness and object-likeness, with features made invariant to domain shifts.
• A multiple instance learning (MIL) framework is employed where each image is treated as a bag of regions, and the model learns to distinguish true object regions from distractors using the learned objectness knowledge.
• The objectness-aware MIL model jointly optimizes for image-level classification and region-level detection, using objectness scores as priors to suppress false positives.
• The domain-invariant objectness model is trained on both target categories and non-target background objects, making it robust to distractors in complex scenes.
• The final detector is trained end-to-end with a joint loss combining classification and localization objectives, guided by the robust objectness priors.

Experimental results

Research questions

• RQ1 Can domain-invariant objectness knowledge be effectively learned from fully labeled categories to improve detection of novel, weakly labeled categories?
• RQ2 How can objectness knowledge be transferred robustly across domains with distribution shifts between strong and weak categories?
• RQ3 Can objectness-aware multiple instance learning reduce false positives caused by object parts and contextual distractors in weakly supervised detection?
• RQ4 Does the proposed method outperform existing mixed supervised detection approaches on benchmark datasets like ILSVRC2013 and PASCAL VOC?
• RQ5 What are the failure modes of the method, and how do they relate to scene complexity and multi-object co-occurrence?

Key findings

• The proposed method achieves state-of-the-art performance on the ILSVRC2013 detection dataset, outperforming existing MSD methods.
• On the PASCAL VOC 2007 test set, the method significantly reduces false positives from object parts such as 'cat face' and 'boat with water' compared to baseline WSD models.
• The domain-invariant objectness model produces fewer object parts in top-15% regions than standard objectness models, due to better background and non-target object modeling.
• The method shows superior generalization to novel categories, even when they are not in the training distribution of the strong categories.
• Failure cases are primarily due to overlapping objects (e.g., 'chair' with 'table', 'person' with 'bicycle') or ambiguous object boundaries, which are inherent challenges in weakly supervised settings.
• The ablation study confirms that domain-invariant objectness learning is critical for robustness, as removing it leads to significant performance drops on distractor-heavy images.