[Paper Review] Octree guided CNN with Spherical Kernels for 3D Point Clouds

Huan Lei, Naveed Akhtar+1 more|arXiv (Cornell University)|Feb 28, 2019

3D Shape Modeling and Analysis39 references18 citations

TL;DR

This paper proposes a novel octree-guided CNN with spherical convolutional kernels for 3D point cloud processing, enabling efficient, translation-invariant, and asymmetric feature learning without dynamic kernel generation. The method achieves state-of-the-art performance on ShapeNet and RueMonge2014 for 3D classification and semantic segmentation by leveraging hierarchical space partitioning and bin-based kernel weighting.

ABSTRACT

We propose an octree guided neural network architecture and spherical convolutional kernel for machine learning from arbitrary 3D point clouds. The network architecture capitalizes on the sparse nature of irregular point clouds, and hierarchically coarsens the data representation with space partitioning. At the same time, the proposed spherical kernels systematically quantize point neighborhoods to identify local geometric structures in the data, while maintaining the properties of translation-invariance and asymmetry. We specify spherical kernels with the help of network neurons that in turn are associated with spatial locations. We exploit this association to avert dynamic kernel generation during network training that enables efficient learning with high resolution point clouds. The effectiveness of the proposed technique is established on the benchmark tasks of 3D object classification and segmentation, achieving new state-of-the-art on ShapeNet and RueMonge2014 datasets.

Motivation & Objective

• Address the challenge of applying standard CNNs to irregular 3D point clouds, which lack regular grid structure.
• Overcome limitations of existing methods such as high memory usage, dynamic kernel generation, and expensive K-NN/range searches.
• Develop a geometrically meaningful convolutional kernel that preserves translation-invariance and asymmetry in 3D space.
• Enable efficient processing of high-resolution point clouds by replacing iterative neighborhood searches with octree-based spatial partitioning.
• Achieve state-of-the-art performance on benchmark 3D point cloud tasks without relying on normal computation or complex graph construction.

Proposed method

• Proposes spherical convolutional kernels that partition the 3D space around each point into volumetric bins, each associated with a learnable weight matrix.
• Uses octree partitioning to hierarchically coarsen the point cloud, with each network layer corresponding to a level in the octree structure.
• Associates each network neuron with a spatial location in the octree, eliminating the need for dynamic kernel generation during training.
• Applies spherical kernels asymmetrically to neighboring points, mimicking standard 2D CNN behavior for compact representation learning.
• Constructs point neighborhoods directly from octree structure, avoiding K-NN and range search, which reduces computational cost.
• Employs a multi-layer network architecture (Ψ-CNN) where feature maps are updated via spherical convolutions at each octree level, progressively reducing resolution.

Experimental results

Research questions

• RQ1 Can a spherical convolutional kernel be designed to maintain translation-invariance and asymmetry in 3D point clouds while enabling efficient learning?
• RQ2 Can octree-based space partitioning replace K-NN or range search for neighborhood computation in 3D CNNs, improving efficiency?
• RQ3 Does associating kernels with fixed spatial locations in the octree eliminate the need for dynamic kernel generation during training?
• RQ4 Can this approach achieve state-of-the-art performance on 3D point cloud classification and segmentation tasks?
• RQ5 How does the proposed method scale in terms of inference time and memory usage compared to existing methods like PointNet++ and OctNet?

Key findings

• The proposed Ψ-CNN achieves 86.8% accuracy on ModelNet40 for 3D object classification, surpassing previous methods including PointNet++.
• On the ShapeNet part segmentation benchmark, the method achieves 84.7% mIoU, setting a new state-of-the-art performance.
• For semantic segmentation on the real-world RueMonge2014 dataset, the method attains 83.5% overall accuracy and 63.6% mean IoU, outperforming OctNet with 20 parametric layers.
• Per-sample test time for a 10K-point cloud is 34.1ms (3.5ms for octree construction, 30.6ms for forward pass), demonstrating efficiency despite high resolution.
• The method avoids normal computation entirely, yet achieves competitive results, indicating robustness to input features.
• Timings show that octree-based neighborhood computation is significantly faster than K-NN and range search for large point clouds, especially at scale.