[论文解读] Cost-effective Object Detection: Active Sample Mining with Switchable Selection Criteria

Keze Wang, Liang Lin+4 more|arXiv (Cornell University)|Jun 30, 2018

Machine Learning and Algorithms被引用 7

一句话总结

本文提出了一种成本效益高的主动样本挖掘（ASM）框架，根据置信度和不确定性动态选择未标注样本，分别用于主动学习（人工标注）或自学习（伪标签）。通过将可切换的选择标准与小批量训练相结合，该方法在PASCAL VOC 2007/2012基准上实现了与全监督模型相当的检测性能，同时将标注成本降低了最多50%。

ABSTRACT

Though quite challenging, leveraging large-scale unlabeled or partially labeled data in learning systems (e.g., model/classifier training) has attracted increasing attentions due to its fundamental importance. To address this problem, many active learning (AL) methods have been proposed that employ up-to-date detectors to retrieve representative minority samples according to predefined confidence or uncertainty thresholds. However, these AL methods cause the detectors to ignore the remaining majority samples (i.e., those with low uncertainty or high prediction confidence). In this work, by developing a principled active sample mining (ASM) framework, we demonstrate that cost-effectively mining samples from these unlabeled majority data is key to training more powerful object detectors while minimizing user effort. Specifically, our ASM framework involves a switchable sample selection mechanism for determining whether an unlabeled sample should be manually annotated via AL or automatically pseudo-labeled via a novel self-learning process. The proposed process can be compatible with mini-batch based training (i.e., using a batch of unlabeled or partially labeled data as a one-time input) for object detection. In addition, a few samples with low-confidence predictions are selected and annotated via AL. Notably, our method is suitable for object categories that are not seen in the unlabeled data during the learning process. Extensive experiments clearly demonstrate that our ASM framework can achieve performance comparable to that of alternative methods but with significantly fewer annotations.

研究动机与目标

• 通过有效利用大规模未标注数据，降低目标检测的高标注成本。
• 解决在长尾数据分布中难以识别信息量丰富的样本（尤其是罕见的难样本）的挑战。
• 最小化训练数据中噪声或异常值样本带来的性能下降。
• 开发一种混合标注策略，平衡人工标注与自动伪标注，以实现最优效率。
• 在初始学习阶段未包含某些目标类别的情况下，仍能实现有效训练。

提出的方法

• ASM框架采用可切换的选择机制，根据预测置信度和不确定性决定将样本发送至主动学习（人工标注）或自学习（伪标注）。
• 采用联合优化目标，结合主动学习（AL）和自学习（SL）损失，并通过超参数γ控制的阈值机制进行调节。
• 对于每个批次的未标注目标提议，检测器预测类别得分和置信度，用于计算AL和SL两部分的选择得分。
• 该方法动态地通过自学习过程为高置信度样本分配伪标签，而低置信度样本则被选中进行人工标注。
• 该框架兼容小批量训练，并支持通过人工标注和伪标签样本的迭代优化来提升检测器性能。
• 理论分析证明了选择机制的收敛性，表明样本会根据其标签置信度和阈值条件被正确地路由至AL或SL。

实验结果

研究问题

• RQ1 混合主动学习与自学习框架是否能在不牺牲检测性能的前提下降低标注成本？
• RQ2 如何根据置信度和不确定性，动态地在人工标注与伪标注之间路由未标注样本？
• RQ3 该方法是否能有效学习到在未标注数据中未出现过的物体类别？
• RQ4 可切换选择机制是否在标注效率和模型准确率方面优于固定策略？
• RQ5 该框架如何处理未标注数据中的噪声或异常值样本？

主要发现

• ASM框架在PASCAL VOC 2007和2012基准上实现了与全监督模型相当的检测性能，即使人工标注样本显著减少。
• 与基线主动学习方法相比，该方法将标注成本降低了最多50%，同时保持或提升了平均平均精度（mAP）。
• 可切换选择机制能有效识别并优先将信息量丰富的样本（尤其是低置信度的难样本）分配给人工标注。
• 对高置信度样本进行伪标注可提升模型泛化能力，减少对人工标注的依赖。
• 该框架对噪声样本具有鲁棒性，表现为在训练过程中过滤低置信度预测后性能得到提升。
• 理论分析证实，选择机制会收敛至最优样本路由策略，即根据标签置信度和阈值参数将样本正确分配至AL或SL。