[论文解读] Learning to Fuse Asymmetric Feature Maps in Siamese Trackers

Wencheng Han, Xingping Dong+3 more|arXiv (Cornell University)|Dec 4, 2020

Video Surveillance and Tracking Methods参考文献 51被引用 4

一句话总结

本文提出了一种可学习的非对称卷积模块（ACM），通过学习目标模板和搜索区域之间非对称特征图的相关性，提升了孪生跟踪器中的特征融合效果。与手工设计的DW-XCorr不同，ACM支持大规模数据的端到端训练，整合了边界框大小先验，并在六个基准测试中实现了最先进性能，在LaSOT上的AUC提升达5.8%。

ABSTRACT

Recently, Siamese-based trackers have achieved promising performance in visual tracking. Most recent Siamese-based trackers typically employ a depth-wise cross-correlation (DW-XCorr) to obtain multi-channel correlation information from the two feature maps (target and search region). However, DW-XCorr has several limitations within Siamese-based tracking: it can easily be fooled by distractors, has fewer activated channels, and provides weak discrimination of object boundaries. Further, DW-XCorr is a handcrafted parameter-free module and cannot fully benefit from offline learning on large-scale data. We propose a learnable module, called the asymmetric convolution (ACM), which learns to better capture the semantic correlation information in offline training on large-scale data. Different from DW-XCorr and its predecessor(XCorr), which regard a single feature map as the convolution kernel, our ACM decomposes the convolution operation on a concatenated feature map into two mathematically equivalent operations, thereby avoiding the need for the feature maps to be of the same size (width and height)during concatenation. Our ACM can incorporate useful prior information, such as bounding-box size, with standard visual features. Furthermore, ACM can easily be integrated into existing Siamese trackers based on DW-XCorror XCorr. To demonstrate its generalization ability, we integrate ACM into three representative trackers: SiamFC, SiamRPN++, and SiamBAN. Our experiments reveal the benefits of the proposed ACM, which outperforms existing methods on six tracking benchmarks. On the LaSOT test set, our ACM-based tracker obtains a significant improvement of 5.8% in terms of success (AUC), over the baseline.

研究动机与目标

• 为解决孪生跟踪器中手工设计的DW-XCorr存在的局限性，如对干扰物敏感以及边界判别能力弱的问题。
• 通过用可学习替代品替代无参数的DW-XCorr，实现相关模块的端到端学习。
• 在无需进行尺寸对齐的情况下，实现不同空间尺寸的非对称特征图的融合。
• 将非视觉先验信息（如边界框大小）整合到相关学习过程中。
• 在不牺牲速度的前提下，提升在多样化基准测试中的跟踪鲁棒性与准确性。

提出的方法

• 提出一种非对称卷积（ACM），将互相关分解为两种数学等价操作，从而实现对不同空间尺寸特征图的融合。
• 设计一种可学习模块，替代孪生跟踪器中的DW-XCorr，支持在大规模数据上进行离线预训练，以提升特征表示能力。
• 引入一种机制，在相关层中将边界框大小作为附加先验信息与视觉特征进行融合。
• 采用双分支结构，模板和搜索区域的特征分别处理后，再通过ACM进行相关计算。
• 使用可学习卷积核作用于拼接后的特征图，避免在拼接时要求特征图尺寸相同。
• 将ACM集成到三种最先进孪生跟踪器（SiamFC、SiamRPN++和SiamBAN）中，仅需极少的架构修改。

Figure 1: Comparison between DW-XCorr and ACM in terms of being fooled by distractors (first row), information distribution across channels (second row) and background suppression to better discriminative target boundaries (third row). DW-XCorr produces similar responses for distractors and the targ

实验结果

研究问题

• RQ1 通过利用大规模预训练，可学习的相关模块是否能在孪生跟踪器中超越手工设计的DW-XCorr？
• RQ2 将边界框大小作为先验信息是否能提升跟踪性能与鲁棒性？
• RQ3 相关模块是否能有效融合非对称特征图，而无需进行空间尺寸对齐？
• RQ4 所提出的ACM是否能减少来自干扰物的误报并改善边界定位？
• RQ5 ACM在LaSOT、VOT和OTB-100等多样化基准测试中，性能提升程度如何？

主要发现

• 在LaSOT基准测试中，基于ACM的跟踪器相比基线模型，成功AUC绝对提升了5.8%，展现出强大的泛化能力。
• 在VOT2016上，SiamBAN-ACM跟踪器相比基线SiamBAN，EAO绝对提升4.4%，且失败率降低了20%。
• 在VOT2019上，ACM-based跟踪器表现出一致的性能提升，相比基线模型具有更高的准确率与鲁棒性。
• 在UAV123上，SiamBAN-ACM达到64.8的AUC得分，优于SiamCAR（61.4）和SiamBAN（63.1）。
• 消融实验证实，AC模块本身以及边界框大小的引入均对性能提升有贡献，完整ACM优于无先验的变体。
• 基于ACM的跟踪器保持了较高的推理速度，SiamFC-ACM在GTX1080Ti上达到172 FPS，速度损失可忽略。

Figure 2: Comparison of (c) AC with (a) DW-XCorr and (b) a naive strategy to fuse different-sized feature maps. (a): DW-XCorr uses a $C$ channel feature map extracted from the template as kernel and convolves instance feature maps in a depth-wise manner to generate a $C$ channel correlation feature