[Paper Review] GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds

Zi‐Hui Zhang, Bo Yang+2 more|arXiv (Cornell University)|May 25, 2023

3D Shape Modeling and Analysis4 citations

TL;DR

GrowSP proposes the first purely unsupervised 3D semantic segmentation method for real-world point clouds, requiring no human annotations or pretrained models. It uses a progressive superpoint growing strategy to learn meaningful semantic primitives through a feature extractor, superpoint constructor, and clustering module, achieving performance close to fully supervised methods on S3DIS and SemanticKITTI datasets.

ABSTRACT

We study the problem of 3D semantic segmentation from raw point clouds. Unlike existing methods which primarily rely on a large amount of human annotations for training neural networks, we propose the first purely unsupervised method, called GrowSP, to successfully identify complex semantic classes for every point in 3D scenes, without needing any type of human labels or pretrained models. The key to our approach is to discover 3D semantic elements via progressive growing of superpoints. Our method consists of three major components, 1) the feature extractor to learn per-point features from input point clouds, 2) the superpoint constructor to progressively grow the sizes of superpoints, and 3) the semantic primitive clustering module to group superpoints into semantic elements for the final semantic segmentation. We extensively evaluate our method on multiple datasets, demonstrating superior performance over all unsupervised baselines and approaching the classic fully-supervised PointNet. We hope our work could inspire more advanced methods for unsupervised 3D semantic learning.

Motivation & Objective

• To address the high cost and impracticality of human-annotated 3D point cloud datasets for semantic segmentation.
• To eliminate dependency on pretrained models or any form of human supervision in 3D semantic segmentation.
• To discover complex semantic classes directly from raw, unstructured point clouds through self-supervised learning.
• To develop a scalable, end-to-end unsupervised pipeline that learns semantic structure without labels.
• To demonstrate that semantic primitives emerge naturally through progressive neighborhood expansion in sparse 3D data.

Proposed method

• A per-point feature extractor based on an untrained neural network (e.g., SparseConv) to learn initial point features from raw point clouds.
• A superpoint constructor that progressively grows local neighborhoods into larger superpoints during training to enhance semantic coherence.
• A progressive training strategy that encourages similar features within growing superpoints while pushing features of different superpoints apart.
• A semantic primitive clustering module that groups superpoints into semantic classes using standard clustering (e.g., K-means) on learned features.
• The method operates entirely in an unsupervised manner, relying only on spatial structure and feature consistency across growing superpoints.
• No pretraining or human-annotated data is used; all learning is driven by the geometric and structural evolution of superpoints.

Experimental results

Research questions

• RQ1 Can 3D semantic segmentation be achieved without any human-annotated labels or pretrained models?
• RQ2 Does progressive superpoint growth enable the discovery of meaningful semantic primitives in raw 3D point clouds?
• RQ3 Can unsupervised clustering of growing superpoints achieve performance competitive with fully supervised methods?
• RQ4 How does the progressive superpoint strategy compare to direct adaptation of unsupervised 2D methods or self-supervised pretraining in 3D?
• RQ5 What is the impact of superpoint size and growth strategy on the quality of learned semantic segmentation?

Key findings

• GrowSP achieves a mean Intersection over Union (mIoU) of 38.3% on the S3DIS dataset, significantly outperforming all unsupervised baselines and approaching the performance of fully supervised PointNet.
• On the SemanticKITTI dataset, GrowSP achieves 14.3% mIoU, surpassing all unsupervised methods and demonstrating strong generalization to large-scale outdoor scenes.
• The method outperforms IIC, PICIE, and their variants by large margins, especially on complex categories like 'car', 'truck', and 'building'.
• The progressive superpoint growth strategy enables the model to discover semantic primitives that are not identifiable at the point level, even in sparse regions.
• GrowSP achieves 76.0% mIoU on the 'wall' category in S3DIS, indicating strong performance on planar and structural elements.
• The ablation study confirms that the progressive growth mechanism is essential, as static superpoints or fixed neighborhood sizes yield significantly worse results.