[论文解读] Graph-DETR3D: Rethinking Overlapping Regions for Multi-View 3D Object Detection

Zehui Chen, Zhenyu Li+4 more|arXiv (Cornell University)|Apr 25, 2022

Advanced Neural Network Applications被引用 5

一句话总结

Graph-DETR3D 提出了一种基于动态图的特征聚合模块，通过提升重叠区域和边界区域的特征学习，增强多视角 3D 目标检测性能，尤其针对被截断的实例。通过图结构学习和深度无关的多尺度训练策略，其在 nuScenes 测试排行榜上实现了 49.5 NDS 的最先进性能。

ABSTRACT

3D object detection from multiple image views is a fundamental and challenging task for visual scene understanding. Due to its low cost and high efficiency, multi-view 3D object detection has demonstrated promising application prospects. However, accurately detecting objects through perspective views in the 3D space is extremely difficult due to the lack of depth information. Recently, DETR3D introduces a novel 3D-2D query paradigm in aggregating multi-view images for 3D object detection and achieves state-of-the-art performance. In this paper, with intensive pilot experiments, we quantify the objects located at different regions and find that the "truncated instances" (i.e., at the border regions of each image) are the main bottleneck hindering the performance of DETR3D. Although it merges multiple features from two adjacent views in the overlapping regions, DETR3D still suffers from insufficient feature aggregation, thus missing the chance to fully boost the detection performance. In an effort to tackle the problem, we propose Graph-DETR3D to automatically aggregate multi-view imagery information through graph structure learning (GSL). It constructs a dynamic 3D graph between each object query and 2D feature maps to enhance the object representations, especially at the border regions. Besides, Graph-DETR3D benefits from a novel depth-invariant multi-scale training strategy, which maintains the visual depth consistency by simultaneously scaling the image size and the object depth. Extensive experiments on the nuScenes dataset demonstrate the effectiveness and efficiency of our Graph-DETR3D. Notably, our best model achieves 49.5 NDS on the nuScenes test leaderboard, achieving new state-of-the-art in comparison with various published image-view 3D object detectors.

研究动机与目标

• 识别并解决多视角 3D 目标检测中的主要性能瓶颈，特别是图像边界处被截断实例的问题。
• 通过用动态 3D 图结构替代单点采样，改进多视角特征聚合。
• 开发一种深度无关的多尺度训练策略，确保数据增强过程中视觉深度一致性。
• 提升多视角监督利用不足的重叠区域中的检测精度。
• 通过可学习的自适应特征聚合机制，在 nuScenes 基准上实现最先进性能。

提出的方法

• 提出动态图特征聚合（DGFA）模块，在目标查询与 2D 特征图之间构建 3D 图，通过图消息传递实现多视角特征融合。
• 利用每个目标查询的参考点在 3D 空间中动态确定 k 个最近邻，形成空间图以实现自适应特征采样。
• 将图节点投影到相机视角以采样 2D 特征，并通过可学习的亲和矩阵融合，优化目标查询表示。
• 引入深度无关的多尺度训练策略，同时缩放图像尺寸和目标深度，以保持深度监督一致性。
• 采用解耦训练变体，以隔离多尺度数据增强对检测头定位性能的影响。
• 以 DETR3D 框架为基线，通过基于图的自适应特征聚合扩展其 3D-2D 查询范式。

实验结果

研究问题

• RQ1 为何图像边界处的被截断实例在多视角 3D 检测中仍是定位误差的主要来源？
• RQ2 与单点采样相比，动态图结构学习是否能提升重叠区域和边界区域的特征聚合性能？
• RQ3 多尺度训练如何影响基于视觉的 3D 检测器中的深度监督与检测头性能？
• RQ4 深度无关的多尺度训练策略是否能缓解标准多尺度数据增强导致的性能下降？
• RQ5 基于图的特征聚合在 nuScenes 等挑战性基准上能将检测精度提升多少？

主要发现

• Graph-DETR3D 在 nuScenes 测试排行榜上达到 49.5 NDS，创下图像级 3D 目标检测的新 SOTA 记录。
• DGFA 模块将 mAP 从单点基线的 29.1 提升至 31.9，证明其在重叠区域具有更优的特征聚合能力。
• 原始多尺度训练因深度监督模糊导致 mAP 下降 2.7，凸显深度无关策略的必要性。
• 深度无关的多尺度训练策略带来最大提升，相比解耦训练提高 0.2 mAP，相比原始训练提高 0.9 mAP。
• 模型在 mAOE 上取得显著提升（从 0.527 降至 0.460），表明定位精度更高，尤其对被截断实例。
• 大量消融实验表明，动态图学习能有效增强多视角可见性有限区域的表征学习。