[Paper Review] Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Clouds

Li Yi, Boqing Gong+1 more|arXiv (Cornell University)|Jul 16, 2020

3D Shape Modeling and Analysis68 references21 citations

TL;DR

This paper proposes a domain adaptation method for LiDAR semantic segmentation by first completing sparse 3D point clouds into dense surfaces using a Sparse Voxel Completion Network (SVCN), then applying semantic labeling on the completed surfaces. The approach achieves 6.0–10.4 absolute mIoU improvement over state-of-the-art domain adaptation methods when transferring between Waymo and nuScenes datasets.

ABSTRACT

We study an unsupervised domain adaptation problem for the semantic labeling of 3D point clouds, with a particular focus on domain discrepancies induced by different LiDAR sensors. Based on the observation that sparse 3D point clouds are sampled from 3D surfaces, we take a Complete and Label approach to recover the underlying surfaces before passing them to a segmentation network. Specifically, we design a Sparse Voxel Completion Network (SVCN) to complete the 3D surfaces of a sparse point cloud. Unlike semantic labels, to obtain training pairs for SVCN requires no manual labeling. We also introduce local adversarial learning to model the surface prior. The recovered 3D surfaces serve as a canonical domain, from which semantic labels can transfer across different LiDAR sensors. Experiments and ablation studies with our new benchmark for cross-domain semantic labeling of LiDAR data show that the proposed approach provides 8.2-36.6% better performance than previous domain adaptation methods.

Motivation & Objective

• Address the domain gap in LiDAR semantic segmentation caused by differences in LiDAR sensor sampling patterns across datasets.
• Overcome the high cost of manual 3D point labeling by enabling transfer learning across different LiDAR configurations.
• Leverage geometric priors in 3D surfaces to improve domain adaptation performance without requiring labeled data for completion.
• Enable domain generalization by training a single surface completion model that generalizes to unseen LiDAR sensor configurations.
• Develop a self-supervised surface completion framework that avoids manual labeling while preserving geometric fidelity for downstream segmentation.

Proposed method

• Use a Sparse Voxel Completion Network (SVCN) to reconstruct complete 3D surfaces from sparse LiDAR point clouds by learning from multi-frame reconstructions.
• Train SVCN using self-supervised supervision from complete surfaces reconstructed across multiple LiDAR sweeps, avoiding the need for manual labeling.
• Introduce local adversarial learning to model surface priors and improve completion quality, especially in occluded or sparse regions.
• Apply a sparse convolutional U-Net for semantic segmentation on the completed 3D surfaces, enabling sensor-agnostic labeling.
• Implement data augmentation during training by randomly sampling polar coordinate bins to simulate diverse LiDAR beam patterns, enhancing domain generalization.
• Use surface completion IoU and Chamfer Distance (CD) as metrics to evaluate completion quality, and mIoU for semantic segmentation performance.

Experimental results

Research questions

• RQ1 Can 3D surface completion effectively reduce domain shift caused by different LiDAR sensor sampling patterns in semantic segmentation?
• RQ2 How does learning from self-supervised surface completion compare to handcrafted sampling alignment methods in cross-domain LiDAR segmentation?
• RQ3 To what extent can a single surface completion network generalize to multiple unseen LiDAR sensor configurations without target-domain fine-tuning?
• RQ4 Does incorporating local adversarial learning improve the quality of 3D surface completion and, consequently, downstream semantic segmentation?
• RQ5 What is the performance gain of the proposed 'Complete & Label' approach over existing domain adaptation and data augmentation baselines in cross-dataset LiDAR segmentation?

Key findings

• The proposed method achieves a 6.0% absolute mIoU improvement over state-of-the-art domain adaptation methods when transferring from Waymo to nuScenes.
• When transferring from nuScenes to Waymo, the method achieves a 10.4% absolute mIoU improvement, demonstrating strong cross-domain generalization.
• The method outperforms handcrafted sampling alignment baselines (B1 and B2) by 4.2–5.6% mIoU, showing the superiority of learning-based completion over interpolation or resampling.
• In domain generalization settings, the generic SVCN trained only on Waymo data achieves 49.8% mIoU on nuScenes-lidarseg and 59.6% on SemanticKITTI, outperforming the no-adaptation baseline by over 7%.
• Surface completion quality, measured by CD and IoU, correlates with segmentation performance: the generalization model achieves 25.7% IoU and 1.800m CD on nuScenes, slightly below the domain-adapted model but still highly effective.
• Ablation studies confirm that local adversarial learning and the use of multi-frame reconstruction significantly improve surface completion and segmentation accuracy.