[论文解读] From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network

Shaoshuai Shi, Zhe Wang+3 more|arXiv (Cornell University)|Jul 8, 2019

Advanced Neural Network Applications参考文献 63被引用 99

一句话总结

引入 Part-A2 net，一种面向 LiDAR 点云的两阶段 3D 目标检测器，它利用对象内部部件位置和 RoI-aware pooling 来提升 3D 检测，在仅使用点云数据的情况下在 KITTI 上达到最先进的结果。

ABSTRACT

3D object detection from LiDAR point cloud is a challenging problem in 3D scene understanding and has many practical applications. In this paper, we extend our preliminary work PointRCNN to a novel and strong point-cloud-based 3D object detection framework, the part-aware and aggregation neural network (Part-$A^2$ net). The whole framework consists of the part-aware stage and the part-aggregation stage. Firstly, the part-aware stage for the first time fully utilizes free-of-charge part supervisions derived from 3D ground-truth boxes to simultaneously predict high quality 3D proposals and accurate intra-object part locations. The predicted intra-object part locations within the same proposal are grouped by our new-designed RoI-aware point cloud pooling module, which results in an effective representation to encode the geometry-specific features of each 3D proposal. Then the part-aggregation stage learns to re-score the box and refine the box location by exploring the spatial relationship of the pooled intra-object part locations. Extensive experiments are conducted to demonstrate the performance improvements from each component of our proposed framework. Our Part-$A^2$ net outperforms all existing 3D detection methods and achieves new state-of-the-art on KITTI 3D object detection dataset by utilizing only the LiDAR point cloud data. Code is available at https://github.com/sshaoshuai/PointCloudDet3D.

研究动机与目标

• 利用来自 3D ground-truth boxes 的自由 intra-object 部分位置监督来学习具有辨识性的 3D 点特征。
• 发展一个两阶段检测器，使用部件信息从点云中提出并细化 3D 框。
• 引入 RoI-aware 点云池化，以保留几何相关信息用于准确的框体细化。

提出的方法

• Two-stage framework with a part-aware stage (Stage-I) and a part-aggregation stage (Stage-II).
• Stage-I learns foreground segmentation and intra-object part locations, and generates 3D proposals via anchor-free or anchor-based strategies.
• Stage-II uses RoI-aware pooling to aggregate part features and employs sparse convolutions for box scoring and refinement.
• Intra-object part locations are defined as relative 3D positions of foreground points within ground-truth boxes and are learned with corresponding losses.
• Anchor-free proposal generation uses a bin-based center regression and residual refinements to predict object centers and orientation.
• Anchor-based proposal generation uses a Region Proposal Network on bird-view features with predefined 3D anchors and a residual-based regression loss.

实验结果

研究问题

• RQ1 如何利用来自 3D 边界框的自由物体内部部位位置信息来提升点云的 3D 目标检测？
• RQ2 仅使用 LiDAR 数据的情况下，具 RoI-aware 池化的两阶段检测器能否超越单阶段及其他两阶段方法？
• RQ3 在基于点云的检测中，3D 提案生成的有效策略是什么（anchor-free 与 anchor-based）？
• RQ4 可微 RoI-aware 池化操作是否提升框得分与定位细化？
• RQ5 Part-A2 net 相较于现有方法，在像 KITTI 这样的标准基准上的表现如何？

主要发现

• Part-A2 net 在 KITTI 上仅使用 LiDAR 点云数据实现了最先进的 3D 检测性能。
• 具部件感知的阶段同时预测物体内部部位位置并生成 3D 提案。
• 通过学习的部件特征，具 RoI-aware 池化的部件聚合阶段提升提案得分和定位细化。
• 两种提案生成策略（anchor-free 与 anchor-based）满足不同部署需求；anchor-free 内存高效，anchor-based 提供更高的召回。
• 该框架在 KITTI 上的运行速度为 14 FPS，截至 2019 年 8 月 15 日，超越了当时的已发表方法。