[Paper Review] SO-Net: Self-Organizing Network for Point Cloud Analysis

Jiaxin Li, Ben M. Chen+1 more|arXiv (Cornell University)|Mar 12, 2018

3D Shape Modeling and AnalysisEngineering18 references96 citations

TL;DR

SO-Net introduces a permutation-invariant architecture for point clouds using a self-organizing map to guide hierarchical feature extraction, achieving competitive accuracy with faster training.

ABSTRACT

This paper presents SO-Net, a permutation invariant architecture for deep learning with orderless point clouds. The SO-Net models the spatial distribution of point cloud by building a Self-Organizing Map (SOM). Based on the SOM, SO-Net performs hierarchical feature extraction on individual points and SOM nodes, and ultimately represents the input point cloud by a single feature vector. The receptive field of the network can be systematically adjusted by conducting point-to-node k nearest neighbor search. In recognition tasks such as point cloud reconstruction, classification, object part segmentation and shape retrieval, our proposed network demonstrates performance that is similar with or better than state-of-the-art approaches. In addition, the training speed is significantly faster than existing point cloud recognition networks because of the parallelizability and simplicity of the proposed architecture. Our code is available at the project website. https://github.com/lijx10/SO-Net

Motivation & Objective

• Motivate efficient and permutation-invariant deep learning on irregular 3D point clouds.
• Model the spatial distribution of points with a self-organizing map (SOM) to guide feature extraction.
• Enable hierarchical, overlapping receptive fields through point-to-node kNN on the SOM.
• Show that the SOM-guided encoding yields competitive classification, segmentation, and autoencoding results.
• Demonstrate faster training compared to state-of-the-art point-cloud networks.

Proposed method

• Build an 8x8 SOM to capture the input point distribution with fixed initial nodes for permutation invariance.
• Perform batch SOM training to ensure permutation-invariant updates.
• For each point, execute k-nearest neighbor search on SOM nodes and normalize points by these nodes.
• Encode mini point clouds attached to each SOM node using shared fully connected layers (PointNet-like) and max-pool to form node features.
• Aggregate node features into a global feature vector for classification, reconstruction, or segmentation via further shared layers.
• Extend the encoder with a decoder for point cloud autoencoding using a two-branch architecture (fully connected and up-convolution) supervised with Chamfer loss.

Experimental results

Research questions

• RQ1 Can a permutation-invariant network effectively model the spatial distribution of point clouds?
• RQ2 Does a self-organizing map enable controllable receptive field overlap and improved local feature aggregation for point clouds?
• RQ3 Can SO-Net achieve competitive performance on classification, segmentation, and reconstruction tasks while offering faster training times?
• RQ4 Does pre-training with a point cloud autoencoder improve downstream tasks?
• RQ5 How robust is SO-Net to variations in SOM size and point/ SOM perturbations?

Key findings

• SO-Net achieves competitive performance on point cloud classification and shape retrieval tasks, with improvements over several baselines in the reported experiments.
• Pre-training with the proposed autoencoder provides a modest boost to ModelNet10 classification (about 0.5% relative gain) and a smaller gain for ModelNet40.
• Training time is substantially faster (e.g., around 3 hours to train on ModelNet40 with a GTX-1080Ti) compared to some state-of-the-art methods.
• SO-Net demonstrates robustness to some SOM and point perturbations, maintaining accuracy under moderate SOM noise and input corruption.
• Increasing the hierarchical layers (repeating the grouping&PN module) yields further accuracy gains on ModelNet40/ModelNet10, though excessive depth can lead to overfitting.