[Paper Review] Label-Efficient Point Cloud Semantic Segmentation: An Active Learning Approach

Xian Shi, Xun Xu+4 more|arXiv (Cornell University)|Jan 18, 2021

3D Shape Modeling and Analysis27 references18 citations

TL;DR

This paper proposes a super-point-based active learning framework for label-efficient 3D point cloud semantic segmentation, using click-based annotation cost measurement and a novel acquisition function combining uncertainty, diversity, and shape-level consistency. It achieves over 95% of fully supervised performance with only 0.8% of the total clicks, outperforming point-level and instance-level selection strategies.

ABSTRACT

Deep learning models are the state-of-the-art methods for semantic point cloud segmentation, the success of which relies on the availability of large-scale annotated datasets. However, it can be extremely time-consuming and prohibitively expensive to compile such datasets. In this work, we propose an active learning approach to maximize model performance given limited annotation budgets. We investigate the appropriate sample granularity for active selection under realistic annotation cost measurement (clicks), and demonstrate that super-point based selection allows for more efficient usage of the limited budget compared to point-level and instance-level selection. We further exploit local consistency constraints to boost the performance of the super-point based approach. We evaluate our methods on two benchmarking datasets (ShapeNet and S3DIS) and the results demonstrate that active learning is an effective strategy to address the high annotation costs in semantic point cloud segmentation.

Motivation & Objective

• To address the high annotation cost of large-scale 3D point cloud semantic segmentation datasets.
• To investigate effective sample selection granularities—point-level, super-point-level, and instance-level—under realistic click-based cost measurement.
• To improve model performance under limited annotation budgets by introducing a shape diversity term and local consistency constraints in the acquisition function.
• To demonstrate that active learning is a viable and efficient strategy for reducing labeling effort in 3D semantic segmentation.

Proposed method

• Proposes super-point-based selection, where geometrically similar points are grouped into super-points to reduce annotation cost per label.
• Introduces a click-based annotation cost metric that reflects real-world labeling effort, replacing traditional point-count metrics.
• Designs a hybrid acquisition function combining uncertainty, feature diversity, and a novel shape diversity term to guide informative sample selection.
• Applies a local consistency loss to enforce consistent predictions within each super-point, leveraging unlabeled data for improved generalization.
• Uses super-point clustering with and without geometric features to evaluate robustness and noise reduction in grouping.
• Employs a two-stage training pipeline: initial pre-training on weakly labeled data, followed by iterative active learning with model finetuning.

Experimental results

Research questions

• RQ1 Is super-point-based selection more efficient than point-level or instance-level selection under realistic click-based cost measurement?
• RQ2 How does incorporating shape-level diversity into the acquisition function affect model performance with limited annotation budgets?
• RQ3 Can local consistency constraints within super-points improve model generalization using unlabeled data?
• RQ4 What is the optimal number of super-points per shape for balancing annotation efficiency and segmentation accuracy?

Key findings

• Super-point-based selection achieves the highest performance efficiency, outperforming both point-level and instance-level selection under the same click budget.
• The proposed acquisition function with uncertainty, diversity, and shape-level terms significantly improves model accuracy compared to baseline strategies.
• Including geometric features in super-point clustering reduces noise rate from 5.05% to 4.82% at 50 super-points per shape, improving grouping quality.
• Clustering shapes into 500 super-points yields the best mIoU (69.0%) under a 10k-click budget, outperforming configurations with 50 to 1000 super-points.
• With only 200k clicks (0.8% of full annotation cost), the method achieves over 95% of the mIoU of the fully supervised baseline on both ShapeNet and S3DIS datasets.
• Random selection performs surprisingly well as a baseline, outperforming traditional uncertainty and diversity methods in some settings, highlighting unique challenges in 3D point cloud labeling.