[Paper Review] Entropy-based Active Learning for Object Detection with Progressive Diversity Constraint

Jiaxi Wu, Jiaxin Chen+1 more|arXiv (Cornell University)|Apr 17, 2022

Image Processing Techniques and Applications4 citations

TL;DR

This paper proposes a novel hybrid active learning method for object detection that combines instance-level uncertainty and diversity through a two-stage framework: Entropy-based Non-Maximum Suppression (ENMS) for uncertainty estimation and a diverse prototype (DivProto) strategy for progressive intra- and inter-class diversity. The approach achieves state-of-the-art performance on MS COCO and Pascal VOC, significantly outperforming existing methods by balancing informativeness and representativeness while maintaining computational efficiency.

ABSTRACT

Active learning is a promising alternative to alleviate the issue of high annotation cost in the computer vision tasks by consciously selecting more informative samples to label. Active learning for object detection is more challenging and existing efforts on it are relatively rare. In this paper, we propose a novel hybrid approach to address this problem, where the instance-level uncertainty and diversity are jointly considered in a bottom-up manner. To balance the computational complexity, the proposed approach is designed as a two-stage procedure. At the first stage, an Entropy-based Non-Maximum Suppression (ENMS) is presented to estimate the uncertainty of every image, which performs NMS according to the entropy in the feature space to remove predictions with redundant information gains. At the second stage, a diverse prototype (DivProto) strategy is explored to ensure the diversity across images by progressively converting it into the intra-class and inter-class diversities of the entropy-based class-specific prototypes. Extensive experiments are conducted on MS COCO and Pascal VOC, and the proposed approach achieves state of the art results and significantly outperforms the other counterparts, highlighting its superiority.

Motivation & Objective

• To address the high annotation cost in object detection by developing an active learning strategy that selects more informative unlabeled images.
• To jointly optimize instance-level uncertainty and cross-image diversity in a computationally efficient manner.
• To improve model performance with fewer labeled samples by ensuring balanced and representative image acquisition.
• To overcome limitations of existing uncertainty-based and diversity-based methods by integrating both principles at the instance level.

Proposed method

• Introduces Entropy-based Non-Maximum Suppression (ENMS) to estimate image-level uncertainty by suppressing redundant object instances based on entropy in the feature space.
• Applies ENMS per image to refine instance-level uncertainty, where higher entropy after suppression indicates higher selection priority.
• Develops a diverse prototype (DivProto) strategy that aggregates class-specific instances into prototypes to enforce intra-class and inter-class diversity.
• Uses entropy-based class-specific prototypes to progressively select images that enhance both diversity within and across object categories.
• Implements a two-stage pipeline: first ENMS for uncertainty filtering, then DivProto for diversity-aware image selection.
• Balances computational cost by performing diversity constraints at the prototype level rather than on all pairwise instance comparisons.

Experimental results

Research questions

• RQ1 How can instance-level uncertainty and diversity be effectively combined in active learning for object detection?
• RQ2 Can a two-stage framework that separates uncertainty estimation and diversity enforcement improve performance while reducing computational cost?
• RQ3 To what extent does progressive diversity at the prototype level enhance model generalization and label efficiency?
• RQ4 How does the proposed method compare to uncertainty-only and diversity-only baselines in terms of accuracy and sample efficiency?

Key findings

• The proposed method achieves state-of-the-art performance on both MS COCO and Pascal VOC benchmarks, outperforming existing active learning methods for object detection.
• At the 30% annotation cycle on MS COCO, the method achieves the highest mean average precision (mAP) with optimal hyper-parameters α=0.5 and β=0.75 in the DivProto module.
• The combination of ENMS and DivProto reduces the standard deviation of instance counts across 80 COCO classes, indicating improved inter-class balance in the selected subset.
• Computational efficiency is significantly improved: the method’s acquisition time (1,015s) is over 99% less than the upper bound of raw instance-level diversity computation (9.95×10⁷s).
• The method outperforms both random sampling and basic entropy-based selection, demonstrating consistent gains across different backbone networks, including ResNet-101.
• Visualization via t-SNE confirms that DivProto produces more representative and diverse prototypes than the entropy baseline, with higher standard deviation across classes.