[Paper Review] SSPC-Net: Semi-supervised Semantic 3D Point Cloud Segmentation Network

Mingmei Cheng, Le Hui+2 more|arXiv (Cornell University)|Apr 16, 2021

3D Shape Modeling and Analysis4 citations

TL;DR

SSPC-Net is a semi-supervised 3D point cloud segmentation network that reduces annotation cost by leveraging few labeled points and generating high-quality pseudo labels for unlabeled points. It uses superpoint graphs with dynamic label propagation and a superpoint dropout strategy to refine pseudo labels, while a coupled attention mechanism enhances feature discrimination, achieving state-of-the-art performance with only 0.01% labeled points.

ABSTRACT

Point cloud semantic segmentation is a crucial task in 3D scene understanding. Existing methods mainly focus on employing a large number of annotated labels for supervised semantic segmentation. Nonetheless, manually labeling such large point clouds for the supervised segmentation task is time-consuming. In order to reduce the number of annotated labels, we propose a semi-supervised semantic point cloud segmentation network, named SSPC-Net, where we train the semantic segmentation network by inferring the labels of unlabeled points from the few annotated 3D points. In our method, we first partition the whole point cloud into superpoints and build superpoint graphs to mine the long-range dependencies in point clouds. Based on the constructed superpoint graph, we then develop a dynamic label propagation method to generate the pseudo labels for the unsupervised superpoints. Particularly, we adopt a superpoint dropout strategy to dynamically select the generated pseudo labels. In order to fully exploit the generated pseudo labels of the unsupervised superpoints, we furthermore propose a coupled attention mechanism for superpoint feature embedding. Finally, we employ the cross-entropy loss to train the semantic segmentation network with the labels of the supervised superpoints and the pseudo labels of the unsupervised superpoints. Experiments on various datasets demonstrate that our semi-supervised segmentation method can achieve better performance than the current semi-supervised segmentation method with fewer annotated 3D points. Our code is available at https://github.com/MMCheng/SSPC-Net.

Motivation & Objective

• To address the high cost and labor intensity of annotating large-scale 3D point clouds for semantic segmentation.
• To improve semi-supervised 3D point cloud segmentation by reducing reliance on fully supervised data.
• To develop a method that accurately infers pseudo labels for unlabeled points using long-range dependencies and uncertainty-aware refinement.
• To enhance feature learning by jointly modeling supervised and extended (pseudo-labeled) superpoints through a coupled attention mechanism.

Proposed method

• The method partitions 3D point clouds into superpoints using geometric similarity to reduce complexity and enable graph-based reasoning.
• It constructs a superpoint graph to model long-range dependencies between superpoints using a gated graph neural network for feature embedding.
• A dynamic label propagation mechanism propagates labels from supervised superpoints to adjacent, semantically similar superpoints along graph edges.
• A superpoint dropout strategy prunes low-confidence pseudo-labeled superpoints during propagation to improve label quality.
• A coupled attention mechanism alternately attends to supervised and extended superpoints to boost feature discrimination and reduce uncertainty.
• The network is trained using a combined cross-entropy loss on both real labels and refined pseudo labels.

Experimental results

Research questions

• RQ1 Can dynamic label propagation on superpoint graphs effectively generate high-quality pseudo labels for unlabeled 3D point cloud regions?
• RQ2 How does incorporating a superpoint dropout strategy improve the reliability of pseudo labels in semi-supervised 3D segmentation?
• RQ3 To what extent can a coupled attention mechanism enhance feature learning by jointly modeling supervised and pseudo-labeled superpoints?
• RQ4 Does the proposed method achieve superior segmentation performance with significantly fewer annotated points compared to existing semi-supervised approaches?

Key findings

• With only 0.01% labeled points, SSPC-Net achieves a mean Intersection over Union (mIoU) of 51.5% on Area5 of the S3DIS dataset, outperforming current state-of-the-art methods.
• The overall accuracy (OA) of extended (pseudo-labeled) superpoints reaches 90.9% at a 0.01% supervision rate, confirming the high quality of generated pseudo labels.
• The ablation study shows that adding dynamic label propagation and coupled attention improves mIoU by 3.1 and 4.5 points respectively, demonstrating their effectiveness.
• The model maintains strong performance even at ultra-low supervision rates, achieving 43.0% mIoU with just 0.002% labeled points.
• The optimal interval for dynamic label propagation is M=40 epochs, where the model achieves the best balance between label refinement and training stability.
• The proportion of extended superpoints increases significantly when supervision is sparse, highlighting the method’s reliance on and effectiveness in pseudo-labeling under low annotation budgets.