[论文解读] Advanced Feature Learning on Point Clouds using Multi-resolution Features and Learnable Pooling

Kevin Tirta Wijaya, Dong-Hee Paek+1 more|arXiv (Cornell University)|May 20, 2022

3D Shape Modeling and Analysis被引用 21

一句话总结

该论文提出 PointStack，一种点云网络，使用多分辨率特征和可学习池化以保留粒度和非最大信息，在 ScanObjectNN 和 ShapeNetPart 上实现了最先进的结果。

ABSTRACT

Existing point cloud feature learning networks often incorporate sequences of sampling, neighborhood grouping, neighborhood-wise feature learning, and feature aggregation to learn high-semantic point features that represent the global context of a point cloud. Unfortunately, the compounded loss of information concerning granularity and non-maximum point features due to sampling and max pooling could adversely affect the high-semantic point features from existing networks such that they are insufficient to represent the local context of a point cloud, which in turn may hinder the network in distinguishing fine shapes. To cope with this problem, we propose a novel point cloud feature learning network, PointStack, using multi-resolution feature learning and learnable pooling (LP). The multi-resolution feature learning is realized by aggregating point features of various resolutions in the multiple layers, so that the final point features contain both high-semantic and high-resolution information. On the other hand, the LP is used as a generalized pooling function that calculates the weighted sum of multi-resolution point features through the attention mechanism with learnable queries, in order to extract all possible information from all available point features. Consequently, PointStack is capable of extracting high-semantic point features with minimal loss of information concerning granularity and non-maximum point features. Therefore, the final aggregated point features can effectively represent both global and local contexts of a point cloud. In addition, both the global structure and the local shape details of a point cloud can be well comprehended by the network head, which enables PointStack to advance the state-of-the-art of feature learning on point clouds. The codes are available at https://github.com/kaist-avelab/PointStack.

研究动机与目标

• 在点云网络中强调在池化和下采样过程中保留粒度信息与非最大点信息的必要性。
• 引入一个多分辨率特征学习框架，聚合来自多个层/分辨率的特征。
• 提出基于多头注意力的可学习池化机制（LP），以替代最大池化并具备置换不变性。
• 证明在形状分类和部件分割任务上，相较于现有基于MLP的及其他架构，性能有所提升。

提出的方法

• 以四个残差块作为骨干网络，生成多分辨率点特征。
• 从每个残差块收集并池化特征，形成堆叠的多分辨率表示。
• 应用可学习池化（LP）模块，使用带可学习查询的多头注意力聚合特征。
• 在多分辨率池化之前，使用单分辨率池化使各分辨率的特征数量标准化。
• 任务头网络为简单的 MLP，用于对 LP 池化后的特征进行分类或分割处理。

实验结果

研究问题

• RQ1 多分辨率特征学习是否能够在保持高语义内容的同时保留点云的粒度信息？
• RQ2 在单分辨率和多分辨率场景中，可学习池化是否提供比最大池化更能保留信息的聚合？
• RQ3 与现有点云网络相比，多分辨率特征和 LP 是否在全局任务（分类）和局部上下文任务（分割）上提升性能？

主要发现

• PointStack 在 ScanObjectNN 上优于此前基于 MLPO 的网络，OA 提升 1.5%，mAcc 提升 1.9%。
• 在 ShapeNetPart 上，PointStack 在实例 mIoU 上超过现有网络 0.4%。
• 消融实验中，多分辨率特征单独提供适度增益，单分辨率 LP 提供更大增益，完整的多分辨率 LP 取得最佳结果。
• PointStack 在 ModelNet40 上实现 93.3% OA，在 ScanObjectNN 上实现 89.6% mAcc，显示在合成与真实数据上的强劲表现。
• PointStack 在 LP 条件下在 ScanObjectNN 上实现 86.9% OA 与 85.8% mAcc，在 ShapeNetPart 上实现 87.2%Inst. mIoU，显示全局与局部上下文理解的平衡表现。