[Paper Review] Localization-Aware Active Learning for Object Detection

Chieh-Chi Kao, Teng‐Yok Lee+2 more|arXiv (Cornell University)|Jan 16, 2018

Machine Learning and Algorithms8 citations

TL;DR

This paper proposes a localization-aware active learning framework for object detection that improves data efficiency by selecting the most informative unlabeled images using two novel metrics: localization tightness (LT) and localization stability (LS). By incorporating both classification and localization uncertainty into the active learning loop, the method reduces the required labeled training data by up to 25% compared to baseline active learning and achieves 96.5% and 81.9% relative improvements on PASCAL VOC 2007 and 2012, respectively.

ABSTRACT

Active learning - a class of algorithms that iteratively searches for the most informative samples to include in a training dataset - has been shown to be effective at annotating data for image classification. However, the use of active learning for object detection is still largely unexplored as determining informativeness of an object-location hypothesis is more difficult. In this paper, we address this issue and present two metrics for measuring the informativeness of an object hypothesis, which allow us to leverage active learning to reduce the amount of annotated data needed to achieve a target object detection performance. Our first metric measures 'localization tightness' of an object hypothesis, which is based on the overlapping ratio between the region proposal and the final prediction. Our second metric measures 'localization stability' of an object hypothesis, which is based on the variation of predicted object locations when input images are corrupted by noise. Our experimental results show that by augmenting a conventional active-learning algorithm designed for classification with the proposed metrics, the amount of labeled training data required can be reduced up to 25%. Moreover, on PASCAL 2007 and 2012 datasets our localization-stability method has an average relative improvement of 96.5% and 81.9% over the baseline method using classification only.

Motivation & Objective

• To address the lack of effective active learning methods for object detection, which traditionally focus only on classification uncertainty.
• To develop quantitative metrics that assess localization uncertainty without requiring ground-truth bounding boxes.
• To reduce the cost and effort of data annotation by selecting the most informative images for labeling.
• To demonstrate that considering both classification and localization uncertainty leads to better model performance with fewer labeled examples.
• To validate the proposed method on standard benchmarks like PASCAL VOC 2007 and 2012, showing significant gains over existing approaches.

Proposed method

• Proposes Localization Tightness (LT) as a metric based on the overlap ratio between region proposals and final detection boxes, estimating how tightly the detector localizes objects.
• Introduces Localization Stability (LS) as a metric that evaluates detector robustness by measuring bounding box variation under Gaussian noise corruption of input images.
• Integrates LT and LS into an active learning pipeline that selects unlabeled images for annotation based on combined uncertainty in both classification and localization.
• Uses a baseline object detector (e.g., Faster R-CNN) to generate predictions and compute uncertainty scores for unlabeled images.
• Applies the uncertainty scores to rank and prioritize images for human annotation, iteratively improving the detector.
• Employs a propose-then-classify pipeline to estimate localization tightness, making it applicable to detectors with explicit region proposal stages.

Experimental results

Research questions

• RQ1 Can localization uncertainty be quantitatively measured in object detection without access to ground-truth bounding boxes?
• RQ2 How does incorporating localization uncertainty improve active learning performance in object detection compared to classification-only uncertainty?
• RQ3 To what extent do the proposed metrics—localization tightness and localization stability—reduce the number of labeled images required for training?
• RQ4 Does the proposed method outperform passive learning and classification-only active learning baselines on standard object detection benchmarks?
• RQ5 How robust are the proposed metrics to noise and variations in region proposal quality?

Key findings

• The proposed method reduces the number of labeled images required to achieve a target detection performance by up to 25% compared to passive learning.
• On the PASCAL VOC 2007 dataset, the localization-stability-based method (LS+C) achieves a 96.5% relative improvement over the classification-only baseline.
• On the PASCAL VOC 2012 dataset, the LS+C method achieves an 81.9% relative improvement over the classification-only baseline.
• The combination of localization tightness and classification uncertainty (LT+C) reduces annotation cost by 12–22% compared to the baseline active learning method.
• The method remains effective even when ground-truth localization is unavailable, as the estimated localization tightness correlates reasonably well with the true value, especially when proposals are close to final detections.
• The computational overhead of computing the metrics is small compared to the time saved in annotation, making the approach cost-effective despite the added complexity.