[Paper Review] Advanced Feature Learning on Point Clouds using Multi-resolution Features and Learnable Pooling

Kevin Tirta Wijaya, Dong-Hee Paek+1 more|arXiv (Cornell University)|May 20, 2022

3D Shape Modeling and Analysis21 citations

TL;DR

The paper proposes PointStack, a point cloud network that uses multi-resolution features and learnable pooling to preserve granularity and non-maximum information, achieving state-of-the-art results on ScanObjectNN and ShapeNetPart.

ABSTRACT

Existing point cloud feature learning networks often incorporate sequences of sampling, neighborhood grouping, neighborhood-wise feature learning, and feature aggregation to learn high-semantic point features that represent the global context of a point cloud. Unfortunately, the compounded loss of information concerning granularity and non-maximum point features due to sampling and max pooling could adversely affect the high-semantic point features from existing networks such that they are insufficient to represent the local context of a point cloud, which in turn may hinder the network in distinguishing fine shapes. To cope with this problem, we propose a novel point cloud feature learning network, PointStack, using multi-resolution feature learning and learnable pooling (LP). The multi-resolution feature learning is realized by aggregating point features of various resolutions in the multiple layers, so that the final point features contain both high-semantic and high-resolution information. On the other hand, the LP is used as a generalized pooling function that calculates the weighted sum of multi-resolution point features through the attention mechanism with learnable queries, in order to extract all possible information from all available point features. Consequently, PointStack is capable of extracting high-semantic point features with minimal loss of information concerning granularity and non-maximum point features. Therefore, the final aggregated point features can effectively represent both global and local contexts of a point cloud. In addition, both the global structure and the local shape details of a point cloud can be well comprehended by the network head, which enables PointStack to advance the state-of-the-art of feature learning on point clouds. The codes are available at https://github.com/kaist-avelab/PointStack.

Motivation & Objective

• Motivate the need to preserve granular and non-maximum point information during pooling and downsampling in point cloud networks.
• Introduce a multi-resolution feature learning framework that aggregates features from multiple layers/resolutions.
• Propose a permutation-invariant learnable pooling mechanism based on multi-head attention to replace max pooling.
• Demonstrate improved performance on shape classification and part segmentation tasks over existing MLP-based and other architectures.

Proposed method

• Employ four residual blocks as the backbone to generate multi-resolution point features.
• Collect and pool features from each residual block to form a stacked multi-resolution representation.
• Apply a learnable pooling (LP) module using multi-head attention with learnable queries to aggregate features.
• Use single-resolution pooling to standardize the number of features across resolutions before multi-resolution pooling.
• Head networks for tasks are simple MLPs that process the LP-pooled features for classification or segmentation.

Experimental results

Research questions

• RQ1 Can multi-resolution feature learning preserve granularity while maintaining high semantic content for point clouds?
• RQ2 Does learnable pooling provide a more information-preserving aggregation than max pooling in both single-resolution and multi-resolution contexts?
• RQ3 Do multi-resolution features and LP improve performance on global tasks (classification) and local-context tasks (segmentation) compared to existing point-cloud nets?

Key findings

• PointStack outperforms prior MLPO-based networks on ScanObjectNN by 1.5% (OA) and 1.9% (mAcc).
• On ShapeNetPart, PointStack exceeds existing networks by 0.4% in instance mIoU.
• In ablations, multi-resolution features alone give modest gains, single-resolution LP provides larger gains, and full multi-resolution LP yields the best results.
• PointStack achieves 93.3% OA on ModelNet40 and 89.6% mAcc on ScanObjectNN, illustrating strong performance on synthetic and real-world data.
• PointStack achieves 86.9% OA and 85.8% mAcc on ScanObjectNN with LP, and 87.2%Inst. mIoU on ShapeNetPart, indicating balanced global and local context understanding.