[论文解读] Learning Auxiliary Monocular Contexts Helps Monocular 3D Object Detection

Xianpeng Liu, Nan Xue+1 more|arXiv (Cornell University)|Dec 8, 2021

Advanced Neural Network Applications被引用 7

一句话总结

该论文提出MonoCon，一种简单而有效的单目3D目标检测方法，通过在训练过程中利用从3D边界框投影得到的2D监督信号（如角点关键点和偏移向量）作为辅助任务。通过端到端学习这些单目上下文信息，并采用轻量级回归头设计，MonoCon在KITTI基准上实现了最先进性能（汽车类26.33 AP@R40），推理速度高达38.7 fps，优于以往方法，且无需使用LiDAR或深度估计等额外数据。

ABSTRACT

Monocular 3D object detection aims to localize 3D bounding boxes in an input single 2D image. It is a highly challenging problem and remains open, especially when no extra information (e.g., depth, lidar and/or multi-frames) can be leveraged in training and/or inference. This paper proposes a simple yet effective formulation for monocular 3D object detection without exploiting any extra information. It presents the MonoCon method which learns Monocular Contexts, as auxiliary tasks in training, to help monocular 3D object detection. The key idea is that with the annotated 3D bounding boxes of objects in an image, there is a rich set of well-posed projected 2D supervision signals available in training, such as the projected corner keypoints and their associated offset vectors with respect to the center of 2D bounding box, which should be exploited as auxiliary tasks in training. The proposed MonoCon is motivated by the Cramer-Wold theorem in measure theory at a high level. In implementation, it utilizes a very simple end-to-end design to justify the effectiveness of learning auxiliary monocular contexts, which consists of three components: a Deep Neural Network (DNN) based feature backbone, a number of regression head branches for learning the essential parameters used in the 3D bounding box prediction, and a number of regression head branches for learning auxiliary contexts. After training, the auxiliary context regression branches are discarded for better inference efficiency. In experiments, the proposed MonoCon is tested in the KITTI benchmark (car, pedestrain and cyclist). It outperforms all prior arts in the leaderboard on car category and obtains comparable performance on pedestrian and cyclist in terms of accuracy. Thanks to the simple design, the proposed MonoCon method obtains the fastest inference speed with 38.7 fps in comparisons

研究动机与目标

• 解决在不依赖深度、LiDAR或多帧数据的情况下，实现单目图像中精确3D目标定位的挑战。
• 通过利用从3D边界框投影得到的丰富且定义良好的2D监督信号，提升单目3D检测性能。
• 设计一种最小化、可端到端训练的框架，通过辅助上下文学习提升检测性能。
• 验证辅助上下文相较于架构改进（如注意力归一化）的有效性。
• 在保证高精度的同时实现快速推理速度，适用于实时自动驾驶应用。

提出的方法

• 该方法使用标准的ConvNet主干网络从单张RGB图像中提取特征。
• 采用共享的回归头模块来预测关键的3D检测参数：3D中心点、尺寸和方向。
• 引入额外的回归头来预测辅助的单目上下文信息，包括投影后的2D角点关键点、其相对于2D中心点的偏移量以及其他几何信号。
• 辅助头与主检测头一起端到端训练，并在推理阶段被丢弃以保持速度。
• 该设计基于Cramèr–Wold定理，该定理为通过2D测量学习3D结构投影提供了理论依据。
• 辅助上下文使用类别无关的头，而检测任务使用类别相关的头，且在汽车、行人和骑行人类别上进行联合训练。

实验结果

研究问题

• RQ1 从3D边界框中学习投影的2D监督信号是否能提升单目3D检测性能？
• RQ2 不同类型的辅助上下文（如角点、偏移向量）对检测精度的贡献如何？
• RQ3 所提出的辅助学习框架是否优于架构改进（如注意力归一化）？
• RQ4 一种最小化、端到端设计且在推理时丢弃辅助头的方法能否实现高精度与快速推理？
• RQ5 辅助上下文带来的性能增益在不同物体类别（汽车、行人、骑行人）中是否一致？

主要发现

• MonoCon在KITTI汽车类别上达到26.33 AP@R40，优于排行榜上所有先前方法。
• 该方法在中等难度设置下达到19.01 AP@R40，在困难样本上达到15.98 AP@R40。
• 与微调后的增强版MonoDLE模型相比，MonoCon在所有难度级别上平均提升3.37%。
• 消融实验表明，在中等难度下，相比无辅助上下文的基线模型，辅助上下文带来了8.49%的绝对性能提升。
• 模型实现38.7 fps的推理速度，是当前最先进方法中最快的。
• 类别无关的辅助头优于类别相关的头，且在各类别间联合训练可进一步提升整体性能。