[论文解读] Active Learning for Deep Visual Tracking

Di Yuan, Xiaojun Chang+4 more|arXiv (Cornell University)|Oct 17, 2021

Video Surveillance and Tracking Methods被引用 4

一句话总结

本文提出了一种用于深度视觉追踪的主动学习框架，通过选择多样化且具有代表性的视频序列进行训练，降低了标注成本，同时保持了具有竞争力的追踪性能。通过利用多帧协作和最近邻判别方法，该方法提升了样本效率；结合Tversky损失进行边界框回归，使追踪器在有限训练预算下实现了最先进（SOTA）的精度。

ABSTRACT

Convolutional neural networks (CNNs) have been successfully applied to the single target tracking task in recent years. Generally, training a deep CNN model requires numerous labeled training samples, and the number and quality of these samples directly affect the representational capability of the trained model. However, this approach is restrictive in practice, because manually labeling such a large number of training samples is time-consuming and prohibitively expensive. In this paper, we propose an active learning method for deep visual tracking, which selects and annotates the unlabeled samples to train the deep CNNs model. Under the guidance of active learning, the tracker based on the trained deep CNNs model can achieve competitive tracking performance while reducing the labeling cost. More specifically, to ensure the diversity of selected samples, we propose an active learning method based on multi-frame collaboration to select those training samples that should be and need to be annotated. Meanwhile, considering the representativeness of these selected samples, we adopt a nearest neighbor discrimination method based on the average nearest neighbor distance to screen isolated samples and low-quality samples. Therefore, the training samples subset selected based on our method requires only a given budget to maintain the diversity and representativeness of the entire sample set. Furthermore, we adopt a Tversky loss to improve the bounding box estimation of our tracker, which can ensure that the tracker achieves more accurate target states. Extensive experimental results confirm that our active learning-based tracker (ALT) achieves competitive tracking accuracy and speed compared with state-of-the-art trackers on the seven most challenging evaluation benchmarks.

研究动机与目标

• 解决为深度视觉追踪大规模训练数据进行标注所导致的高成本和高时间消耗问题。
• 在固定标注预算下，提升深度卷积神经网络模型的表征能力。
• 通过时间一致性与异常值过滤，确保所选训练样本的多样性与代表性。
• 通过引入Tversky损失函数，提升边界框估计精度，从而增强追踪鲁棒性。
• 证明基于主动学习的样本选择方法可在较少标注样本下实现与随机选择或全数据训练相当的追踪性能。

提出的方法

• 提出一种基于多帧协作的主动学习策略，通过利用视频序列中的时间一致性，选择能最大化多样性的训练序列。
• 提出一种基于平均最近邻距离的最近邻判别方法，用于过滤孤立或低质量序列，确保所选子集的代表性。
• 制定一种样本选择流程，在固定预算下联合优化多样性（通过时间一致性）与代表性（通过异常值检测）。
• 将Tversky损失集成到追踪器的边界框回归头中，通过不对称地惩罚假阳性和假阴性，提升定位精度。
• 在主动选择的视频序列子集上训练基于深度卷积神经网络的追踪器（ALT），实现在最小标注工作量下的高性能。

Figure 1: A visual experimental comparison of the proposed ALT tracker with some state-of-the-art trackers. Thanks to the training samples selected by active learning and the target boundary box estimation improved by Tversky loss, our ALT tracker was able to obtain more accurate tracking results.

实验结果

研究问题

• RQ1 与单帧或随机选择相比，基于多帧协作的主动学习是否能提升视觉追踪所选训练序列的多样性？
• RQ2 基于平均最近邻距离的过滤方法在去除低质量或孤立序列、提升代表性方面是否有效？
• RQ3 集成Tversky损失是否能带来边界框估计和整体追踪精度的可测量提升？
• RQ4 在小规模主动选择的数据子集上训练的追踪器，是否能实现与在完整数据集上训练的SOTA追踪器相当的性能？
• RQ5 主动学习在多大程度上降低了标注成本，同时保持或提升了追踪性能？

主要发现

• 所提出的基于主动学习的追踪器（ALT）在七个基准数据集上取得了最佳EAO得分，优于DiMP50和ATOM等SOTA追踪器。
• 尽管仅在极少数标注序列上进行训练，ALT仍表现出具有竞争力的追踪精度，证明了主动学习策略的有效性。
• 使用Tversky损失显著提升了边界框估计精度，尤其在遮挡和形变等复杂场景下，实现了更准确的目标定位。
• 定性结果表明，ALT在复杂序列（如bird1、jump、soccer）中保持了鲁棒的追踪性能，优于SiamRPN++和PrDiMP50在遮挡和形变情况下的表现。
• 多帧协作与最近邻过滤的结合，确保了在严格预算约束下，所选训练集仍保持高度的多样性和代表性。

Figure 2: An example of training sample selection under different methods: random, single frame-based active learning (SAL), multi-frame cooperation-based active learning (MAL), and neighborhood validation-based re-selection (KMAL). The bold (green) box indicates these samples selected by each of th