[论文解读] Feature Intertwiner for Object Detection

Hongyang Li, Bo Dai+3 more|arXiv (Cornell University)|Mar 28, 2019

Advanced Neural Network Applications参考文献 40被引用 14

一句话总结

本文提出了一种即插即用的特征纠缠模块（feature intertwiner），用于目标检测，通过在高分辨率（可靠）与低分辨率（不可靠）目标特征之间实现相互知识蒸馏，提升特征紧凑性。通过利用最优传输（optimal transport）对齐特征分布，该方法在不使用多尺度测试的情况下，使小目标的mAP提升2.6%，大目标的mAP提升0.8%，显著优于当前最先进检测器。

ABSTRACT

A well-trained model should classify objects with a unanimous score for every category. This requires the high-level semantic features should be as much alike as possible among samples. To achive this, previous works focus on re-designing the loss or proposing new regularization constraints. In this paper, we provide a new perspective. For each category, it is assumed that there are two feature sets: one with reliable information and the other with less reliable source. We argue that the reliable set could guide the feature learning of the less reliable set during training - in spirit of student mimicking teacher behavior and thus pushing towards a more compact class centroid in the feature space. Such a scheme also benefits the reliable set since samples become closer within the same category - implying that it is easier for the classifier to identify. We refer to this mutual learning process as feature intertwiner and embed it into object detection. It is well-known that objects of low resolution are more difficult to detect due to the loss of detailed information during network forward pass (e.g., RoI operation). We thus regard objects of high resolution as the reliable set and objects of low resolution as the less reliable set. Specifically, an intertwiner is designed to minimize the distribution divergence between two sets. The choice of generating an effective feature representation for the reliable set is further investigated, where we introduce the optimal transport (OT) theory into the framework. Samples in the less reliable set are better aligned with aid of OT metric. Incorporated with such a plug-and-play intertwiner, we achieve an evident improvement over previous state-of-the-arts.

研究动机与目标

• 通过减少高维特征空间中的类内差异，提升深度目标检测器的特征紧凑性。
• 解决小尺寸、低分辨率目标在神经网络推理过程中因特征退化而难以检测的问题。
• 提出一种可泛化的即插即用模块，实现可靠（高分辨率）与不可靠（低分辨率）特征集之间的相互特征蒸馏。
• 通过历史缓冲区动态维护类别特定的特征代表，提升模型泛化能力。
• 使用最优传输作为正则化度量，提升特征分布之间的对齐效果。

提出的方法

• 该方法引入一种特征纠缠模块，将高分辨率目标特征视为‘教师’，低分辨率特征视为‘学生’，实现相互特征蒸馏。
• 采用与类别相关的历史缓冲区，在训练批次中存储并更新大目标的代表性特征，以确保稳定的监督信号。
• 使用最优传输（OT）作为差异度量，衡量并最小化可靠与不可靠特征集之间的分布差异。
• 通过轻量级生成器与判别器网络实现纠缠模块，基于OT的损失函数在低维空间中强制对齐特征。
• 将该框架集成到两阶段检测器（如Faster R-CNN）中，参数量与计算开销极低。
• 采用可分离的历史缓冲区更新策略，避免梯度干扰，提升训练稳定性和性能。

实验结果

研究问题

• RQ1 高分辨率与低分辨率目标特征之间的相互知识蒸馏是否能提升特征紧凑性与检测精度？
• RQ2 最优传输与标准距离度量相比，在跨分辨率层级对齐特征分布方面表现如何？
• RQ3 在小批量中缺少大目标时，动态维护可靠特征的历史缓冲区是否能提升训练稳定性和性能？
• RQ4 特征纠缠模块在本质上更具挑战性的小目标上，能多大程度上提升检测性能？
• RQ5 该纠缠模块是否可作为通用模块应用于不同检测架构，而无需修改网络结构？

主要发现

• 特征纠缠模块使小目标的平均平均精度（mAP）提升2.6个百分点，大目标提升0.8个百分点，表明在困难样本上具有显著增益。
• 在COCO数据集上，所提出的InterNet模型在不使用多尺度测试的情况下达到42.5% mAP，优于Mask R-CNN（39.2%）和SSD（31.2%）。
• 使用最优传输进行分布对齐可提升特征聚类效果，t-SNE可视化显示类间混淆与异常点减少。
• 采用‘全部历史’策略与窗口大小为1的历史缓冲区，使mAP从40.1%提升至40.5%，表明其在训练过程中有效缓解类别不平衡问题。
• 该方法仅使推理时间增加5%（325ms vs 308ms），显存增加1.3GB/GPU，表明尽管性能提升显著，但开销极低。
• 模型对噪声标注具有鲁棒性：例如，'couch'类AP下降归因于标注噪声，而非纠缠模块本身，表明该方法在真实世界数据缺陷下仍具稳定性。