[论文解读] Object Detection with Transformers: A Review

Tahira Shehzadi, Khurram Azeem Hashmi+2 more|arXiv (Cornell University)|Jun 7, 2023

Advanced Neural Network Applications被引用 17

一句话总结

本文综述了 DETR 及其后续的 21 项基于 DETR 的改进，分析架构变动、训练收敛性，以及在 COCO 等基准上的性能。

ABSTRACT

The astounding performance of transformers in natural language processing (NLP) has motivated researchers to explore their applications in computer vision tasks. DEtection TRansformer (DETR) introduces transformers to object detection tasks by reframing detection as a set prediction problem. Consequently, eliminating the need for proposal generation and post-processing steps. Initially, despite competitive performance, DETR suffered from slow training convergence and ineffective detection of smaller objects. However, numerous improvements are proposed to address these issues, leading to substantial improvements in DETR and enabling it to exhibit state-of-the-art performance. To our knowledge, this is the first paper to provide a comprehensive review of 21 recently proposed advancements in the original DETR model. We dive into both the foundational modules of DETR and its recent enhancements, such as modifications to the backbone structure, query design strategies, and refinements to attention mechanisms. Moreover, we conduct a comparative analysis across various detection transformers, evaluating their performance and network architectures. We hope that this study will ignite further interest among researchers in addressing the existing challenges and exploring the application of transformers in the object detection domain. Readers interested in the ongoing developments in detection transformers can refer to our website at: https://github.com/mindgarage-shan/trans_object_detection_survey

研究动机与目标

• 提供对 DETR 及其架构模块的全面概览。
• 按 backbone、预训练、注意力机制和查询设计对 DETR 的改进进行分类和总结。
• 评估检测变换器在 MS COCO 上的性能和训练动态。
• 讨论检测变换器的构建模块及未来发展方向。

提出的方法

• 整理并描述 DETR 及其众多增强。
• 比较各变体的架构组件和训练策略。
• 评估在 COCO 上的收敛行为与目标检测性能。
• 总结不同注意力与查询设计在效率与准确性方面的考量。

Figure 1 : Statistics overview of the literature on Transformers. (a) Number of citations per year of Transformer papers. (b) Citations in the last 12 months on Detection Transformer papers. (c) Modification percentage in original DEtection TRansformer (DETR) to improve performance and training conv

实验结果

研究问题

• RQ1 DETR 的关键架构组件有哪些，它们在后续版本中是如何演变的？
• RQ2 backbone 的变更、预训练策略、注意力机制和查询设计如何影响训练收敛性和小目标检测？
• RQ3 最先进的检测变换器在 MS COCO 上在准确性和效率方面的比较如何？
• RQ4 检测变换器的主要挑战和未来方向是什么？

主要发现

• DETR 引入了一种基于集合的端到端目标检测器，去除了区域提案和 NMS。
• 大量改进解决了收敛慢和小目标性能问题，其中包括可变形注意力和多尺度特征。
• Deformable-DETR 通过将注意力样本聚焦在参考点附近来降低计算复杂度、加速训练。
• 其他变体（UP-DETR、SMCA-DETR、Conditional-DETR、TSP-DETR 等）提出预训练、空间调制共注意、跨注意力改进，以及基于 RoI 的细化，以提高收敛性和准确性。
• WB-DETR 通过在没有 CNN 主干的情况下使用变换器编码器-解码器来进行无 backbone 检测，并通过局部信息增强来弥补局部性缺失。
• 总体而言，调研的方法相比原始 DETR 显示出更快的收敛速度和对小目标性能的改进，同时存在多样的设计取舍。

Figure 2 : An overview of the DEtection TRansformer (DETR) and its modifications proposed by recent methods to improve performance and training convergence. It considers the detection a set prediction task and uses the Transformer to free the network from post-processing steps such as non-maximal su