[Paper Review] A new weakly supervised approach for ALS point cloud semantic segmentation

Puzuo Wang, Wei Yao|arXiv (Cornell University)|Oct 4, 2021

Remote Sensing and LiDAR Applications4 citations

TL;DR

This paper proposes a lightweight, weakly supervised deep learning framework for semantic segmentation of ALS point clouds using only sparse, incomplete labels. By integrating entropy regularization, ensemble-based consistency constraints, and online soft pseudo-labeling, the method significantly improves classification accuracy—achieving 83.0% overall accuracy and 70.0% average F1 score on the ISPRS Vaihingen dataset with just 0.1% of labeled points, outperforming existing weakly supervised methods and approaching full-supervision performance.

ABSTRACT

While there are novel point cloud semantic segmentation schemes that continuously surpass state-of-the-art results, the success of learning an effective model usually rely on the availability of abundant labeled data. However, data annotation is a time-consuming and labor-intensive task, particularly for large-scale airborne laser scanning (ALS) point clouds involving multiple classes in urban areas. Thus, how to attain promising results while largely reducing labeling works become an essential issue. In this study, we propose a deep-learning based weakly supervised framework for semantic segmentation of ALS point clouds, exploiting potential information from unlabeled data subject to incomplete and sparse labels. Entropy regularization is introduced to penalize the class overlap in predictive probability. Additionally, a consistency constraint by minimizing the discrepancy distance between instant and ensemble predictions is designed to improve the robustness of predictions. Finally, we propose an online soft pseudo-labeling strategy to create extra supervisory sources in an efficient and nonpaprametric way. Extensive experimental analysis using three benchmark datasets demonstrates that in case of sparse point annotations, our proposed method significantly boosts the classification performance without compromising the computational efficiency. It outperforms current weakly supervised methods and achieves a comparable result against full supervision competitors. For the ISPRS 3D Labeling Vaihingen data, by using only 0.1% of labels, our method achieves an overall accuracy of 83.0% and an average F1 score of 70.0%, which have increased by 6.9% and 12.8% respectively, compared to model trained by sparse label information only.

Motivation & Objective

• To reduce the heavy annotation burden in large-scale ALS point cloud semantic segmentation.
• To improve classification performance under weak supervision using only sparse, incomplete labels.
• To develop a lightweight, efficient framework compatible with existing deep learning networks for point cloud segmentation.
• To explore the potential of unlabeled data and predictive confidence to enhance model robustness and generalization.

Proposed method

• Introduces entropy regularization to penalize class overlap in predictive probability distributions, improving prediction confidence.
• Applies a consistency constraint that minimizes the difference between current model predictions and ensemble predictions across multiple training steps.
• Employs an online soft pseudo-labeling strategy to dynamically generate additional supervisory signals from model predictions on unlabeled data.
• Designs a lightweight framework that can be easily integrated into existing point cloud segmentation networks without architectural overhaul.
• Uses a dual-branch ensemble mechanism to stabilize predictions and reduce uncertainty during training.
• Optimizes the training process through joint optimization of supervised loss on sparse labels and self-supervised losses from unlabeled data.

Experimental results

Research questions

• RQ1 Can semantic segmentation performance on ALS point clouds be significantly improved using only 0.1% of labeled data?
• RQ2 How effective are entropy regularization and consistency constraints in reducing prediction uncertainty under weak supervision?
• RQ3 To what extent can online soft pseudo-labeling enhance model generalization with minimal annotation effort?
• RQ4 How does the proposed method compare to full-supervision baselines when only sparse labels are available?

Key findings

• On the ISPRS Vaihingen dataset, the method achieves 83.0% overall accuracy and 70.0% average F1 score using only 0.1% of labels, a 6.9% and 12.8% improvement over baseline models trained with sparse labels alone.
• The method outperforms existing weakly supervised approaches on all three benchmark datasets: ISPRS, LASDU, and H3D.
• Even with only 0.1% of labels, the method achieves comparable performance to full-supervision models, particularly on dense urban scenes.
• The integration of DualAttention into KPConv did not yield significant performance gains, suggesting that general attention modules may not be sufficient without architecture-specific design.
• The method maintains high computational efficiency, making it suitable for real-world deployment on large-scale ALS point clouds.
• Performance degradation is observed on the ISPRS dataset when contextual information is added, indicating potential limitations in handling sparse, low-density, and high-category scenes.