[Paper Review] ShellNet: Efficient Point Cloud Convolutional Neural Networks using Concentric Shells Statistics

Zhiyuan Zhang, Binh‐Son Hua+1 more|arXiv (Cornell University)|Aug 17, 2019

3D Shape Modeling and AnalysisEngineering28 references51 citations

TL;DR

ShellNet introduces ShellConv, a permutation-invariant convolution on concentric spherical shells, enabling fast, end-to-end learning directly on point clouds and achieving state-of-the-art results in classification and segmentation with lightweight architecture.

ABSTRACT

Deep learning with 3D data has progressed significantly since the introduction of convolutional neural networks that can handle point order ambiguity in point cloud data. While being able to achieve good accuracies in various scene understanding tasks, previous methods often have low training speed and complex network architecture. In this paper, we address these problems by proposing an efficient end-to-end permutation invariant convolution for point cloud deep learning. Our simple yet effective convolution operator named ShellConv uses statistics from concentric spherical shells to define representative features and resolve the point order ambiguity, allowing traditional convolution to perform on such features. Based on ShellConv we further build an efficient neural network named ShellNet to directly consume the point clouds with larger receptive fields while maintaining less layers. We demonstrate the efficacy of ShellNet by producing state-of-the-art results on object classification, object part segmentation, and semantic scene segmentation while keeping the network very fast to train.

Motivation & Objective

• Motivate efficient learning directly on 3D point clouds without point order dependence.
• Propose ShellConv that uses concentric spherical shells to define region-aware, order-invariant features.
• Build ShellNet to achieve large receptive fields with few layers and fast training.
• Demonstrate state-of-the-art performance on object classification, object part segmentation, and semantic scene segmentation.

Proposed method

• Define ShellConv that partitions a local neighborhood into concentric spherical shells.
• Represent each shell by a max-pooled feature over points in that shell.
• Concatenate shell features and apply a 1D convolution from inner to outer shells.
• Use an encoder (three ShellConv layers) for classification and a U-Net-like encoder-decoder for segmentation.
• Train with standard backpropagation; use mlp to lift per-point features within shells.

Experimental results

Research questions

• RQ1 Can a permutation-invariant, shell-based convolution achieve competitive accuracy with fewer parameters and faster training on 3D point clouds?
• RQ2 How does ShellConv’s shell-based aggregation affect receptive field and performance across classification and segmentation tasks?
• RQ3 What is the impact of shell size and neighbor sampling strategy on accuracy and efficiency?

Key findings

• ShellNet achieves state-of-the-art accuracy on ModelNet40 classification with shell sizes around 16.
• ShellNet delivers strong segmentation performance on ShapeNet (part), ScanNet, and S3DIS datasets, often ranking 1st or near 1st in mIoU and OA.
• ShellNet exhibits superior efficiency with fewer parameters and lower FLOPs, delivering faster training and inference than several baselines.
• Increasing shell size enlarges receptive field and can improve accuracy up to an optimal point (e.g., ss=32 gives 93.1% in classification; ss=64 slightly drops to 92.8%).
• The ShellConv operator enables effective local feature learning directly on point clouds while maintaining permutation invariance.