[Paper Review] PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds

Mutian Xu, Runyu Ding+2 more|arXiv (Cornell University)|Mar 26, 2021

3D Shape Modeling and AnalysisEngineering58 references28 citations

TL;DR

PAConv constructs convolution kernels by dynamically assembling weight matrices from a Weight Bank, guided by ScoreNet based on point positions, achieving strong results with simple backbones on 3D point clouds.

ABSTRACT

We introduce Position Adaptive Convolution (PAConv), a generic convolution operation for 3D point cloud processing. The key of PAConv is to construct the convolution kernel by dynamically assembling basic weight matrices stored in Weight Bank, where the coefficients of these weight matrices are self-adaptively learned from point positions through ScoreNet. In this way, the kernel is built in a data-driven manner, endowing PAConv with more flexibility than 2D convolutions to better handle the irregular and unordered point cloud data. Besides, the complexity of the learning process is reduced by combining weight matrices instead of brutally predicting kernels from point positions. Furthermore, different from the existing point convolution operators whose network architectures are often heavily engineered, we integrate our PAConv into classical MLP-based point cloud pipelines without changing network configurations. Even built on simple networks, our method still approaches or even surpasses the state-of-the-art models, and significantly improves baseline performance on both classification and segmentation tasks, yet with decent efficiency. Thorough ablation studies and visualizations are provided to understand PAConv. Code is released on https://github.com/CVMI-Lab/PAConv.

Motivation & Objective

• Address irregularity and unordered nature of 3D point clouds.
• Provide a plug-and-play convolution operator that works with simple backbones.
• Increase modeling flexibility over fixed kernel or brute-force kernel prediction methods.
• Demonstrate state-of-the-art or competitive performance on classification and segmentation benchmarks.

Proposed method

• Define a Weight Bank of multiple basis weight matrices.
• Use ScoreNet to predict coefficients from relative point positions.
• Construct dynamic kernels as a weighted sum of Bank matrices: K(p_i,p_j)=sum_m S_ij^m B_m.
• Train ScoreNet with inputs based on point relations (e.g., coordinates and distances).
• Regularize weight matrices to encourage diversity via a correlation loss L_corr.

Experimental results

Research questions

• RQ1 Can a position-adaptive, dynamically assembled kernel improve point cloud convolutions without heavily engineered backbones?
• RQ2 How does the kernel assembly mechanism compare to brute-force kernel prediction or fixed-kernel point methods in terms of accuracy and efficiency?
• RQ3 What is the impact of ScoreNet inputs, normalization, and weight diversity on performance?
• RQ4 Does PAConv generalize across object-level and scene-level tasks with simple network architectures?

Key findings

• PAConv improves PointNet by 4.0 percentage points and DGCNN by 1.0 point on ModelNet40 without voting.
• On ShapeNet Part, PAConv with DGCNN improves class and instance mIoU by 2.3 and 0.9 points respectively.
• On S3DIS, PAConv with L_corr achieves the best mIoU among block-sampled methods and boosts PointNet++ by 9.31 points.
• PAConv achieves competitive or state-of-the-art performance with notably lower FLOPs than some baselines.
• A correlation-regularized version of PAConv yields additional gains in segmentation accuracy (mIoU) and weight diversity.