[论文解读] Multi Point-Voxel Convolution (MPVConv) for Deep Learning on Point Clouds

Wei Zhou, Xin Cao+4 more|arXiv (Cornell University)|Jul 28, 2021

Robotics and Sensor-Based Localization被引用 4

一句话总结

本文提出多点-体素卷积（MPVConv），一种新颖的3D CNN模块，通过3D CNN和MLP结合基于点和基于体素的特征，增强点特征的邻域聚合能力，同时保持体素特征的独立性。MPVConv将PointNet的准确率提升最高达36%，相较于基于体素的模型实现最高34倍的速度提升，在延迟更低的情况下超越了当前最先进模型。

ABSTRACT

The existing 3D deep learning methods adopt either individual point-based features or local-neighboring voxel-based features, and demonstrate great potential for processing 3D data. However, the point based models are inefficient due to the unordered nature of point clouds and the voxel-based models suffer from large information loss. Motivated by the success of recent point-voxel representation, such as PVCNN, we propose a new convolutional neural network, called Multi Point-Voxel Convolution (MPVConv), for deep learning on point clouds. Integrating both the advantages of voxel and point-based methods, MPVConv can effectively increase the neighboring collection between point-based features and also promote independence among voxel-based features. Moreover, most of the existing approaches aim at solving one specific task, and only a few of them can handle a variety of tasks. Simply replacing the corresponding convolution module with MPVConv, we show that MPVConv can fit in different backbones to solve a wide range of 3D tasks. Extensive experiments on benchmark datasets such as ShapeNet Part, S3DIS and KITTI for various tasks show that MPVConv improves the accuracy of the backbone (PointNet) by up to extbf{36\%}, and achieves higher accuracy than the voxel-based model with up to extbf{34}$ imes$ speedups. In addition, MPVConv outperforms the state-of-the-art point-based models with up to extbf{8}$ imes$ speedups. Notably, our MPVConv achieves better accuracy than the newest point-voxel-based model PVCNN (a model more efficient than PointNet) with lower latency.

研究动机与目标

• 解决基于点的模型在3D点云学习中的效率低下问题，以及基于体素的模型存在的信息损失问题。
• 设计一种通用的3D深度学习模块，支持语义分割和3D检测等多种任务。
• 在统一且高效的架构中整合点表示与体素表示的优势——点表示的邻域聚合能力，体素表示的特征独立性。
• 实现与现有主干网络（如PointNet）的即插即用集成，以广泛适用于各类3D视觉任务。

提出的方法

• MPVConv采用多分支架构，分别通过3D CNN分支和MLP分支处理基于点和基于体素的特征。
• 利用1×1×1 3D卷积层融合并增强点与体素表示之间的跨模态特征交互。
• 融合模块结合来自两个分支的特征，在不显著增加内存或延迟开销的前提下提升特征表示能力。
• 通过在不同分辨率下处理点和体素，实现多尺度特征学习，增强对局部几何结构的理解。
• 利用层次化特征聚合机制，强化基于点特征的局部邻域连接性。
• 该架构设计为模块化，兼容多种主干网络，便于适配不同3D任务。

Figure 1: MPVConv increases neighborhood collection in point-based features and independence in voxel-based features by adopting 3D CNNs and MLPs on both types of features.

实验结果

研究问题

• RQ1 统一的3D卷积模块能否有效结合基于点和基于体素深度学习的优势，以提升性能与效率？
• RQ2 MPVConv如何在增强基于点特征的邻域聚合能力的同时，促进基于体素特征的独立性？
• RQ3 MPVConv在部分分割、室内场景分割和3D检测等多样化3D任务中，能在多大程度上提升准确率与速度？
• RQ4 与PVCNN和F-PointNet等SOTA模型相比，MPVConv在准确率、推理速度和内存效率方面表现如何？
• RQ5 MPVConv能否无缝集成到PointNet等现有主干网络中，以在多个基准测试中提升性能？

主要发现

• 在ShapeNet Part数据集上，MPVConv将PointNet主干网络的准确率最高提升36%，mIoU达到85.76%。
• 在S3DIS数据集上，MPVConv在大规模室内场景分割任务中达到79.85% mIoU，优于F-PVCNN及其他基线模型。
• 在KITTI数据集的3D检测任务中，F-MPVCNN相较F-PointNet的mAP最高提升13.2%，在困难行人类别上相较F-PointNet++提升8.6%。
• MPVConv在保持更高准确率的同时，相较于基于体素的模型实现最高34倍的速度提升，展现出卓越的效率。
• MPVConv中的1×1×1 3D CNN层使mIoU提升0.04%，仅增加0.4ms延迟和0.009GB内存开销。
• 消融实验表明，融合模块以及组合特征组件（V₁ + V₂ + P₂）对性能至关重要，完整MPVConv配置取得最佳结果。

Figure 2: Segmentation results on ShapeNet Part. (a)(c) Ground truth. (b)(d) Our results.