[Paper Review] KPRNet: Improving projection-based LiDAR semantic segmentation

Deyvid Kochanov, Fatemeh Karimi Nejadasl+1 more|arXiv (Cornell University)|Jul 24, 2020

Robotics and Sensor-Based Localization14 references61 citations

TL;DR

KPRNet combines a 2D projection-based CNN with a learnable 3D KPConv module to replace post-processing, achieving state-of-the-art mIoU (63.1) on SemanticKITTI.

ABSTRACT

Semantic segmentation is an important component in the perception systems of autonomous vehicles. In this work, we adopt recent advances in both image and point cloud segmentation to achieve a better accuracy in the task of segmenting LiDAR scans. KPRNet improves the convolutional neural network architecture of 2D projection methods and utilizes KPConv to replace the commonly used post-processing techniques with a learnable point-wise component which allows us to obtain more accurate 3D labels. With these improvements our model outperforms the current best method on the SemanticKITTI benchmark, reaching an mIoU of 63.1.

Motivation & Objective

• Improve LiDAR semantic segmentation by leveraging strong 2D projection-based CNNs pre-trained on urban datasets.
• Eliminate hand-crafted post-processing by introducing a learnable 3D KPConv module for label refinement.
• Achieve higher 3D labeling accuracy by end-to-end learning from projected 2D features to 3D point labels.
• Demonstrate state-of-the-art performance on the SemanticKITTI benchmark compared to existing 2D and point-wise approaches.

Proposed method

• Use a 2D semantic segmentation network with a ResNeXt-101 encoder and a Panoptic-DeepLab-like decoder pre-trained on Cityscapes.
• Project LiDAR scans to 2D range images and extract CNN features, then back-project to 3D points.
• Attach a KPConv-based 3D module to refine per-point labels before final classification.
• Unify the 2D CNN and 3D KPConv into an end-to-end trainable pipeline.
• Train with SGD, cosine learning rate schedule, and standard data augmentations; evaluate on SemanticKITTI.

Experimental results

Research questions

• RQ1 Can a strengthened 2D projection-based CNN improve LiDAR semantic segmentation performance when combined with a learnable 3D component?
• RQ2 Does replacing traditional post-processing (e.g., CRF, KNN) with a KPConv layer yield more accurate 3D labels?
• RQ3 How does the proposed KPRNet perform against state-of-the-art methods on SemanticKITTI in terms of mean IoU?

Key findings

• On SemanticKITTI, KPRNet achieves a mean IoU of 63.1, surpassing the previous best 2D method SalsaNext by 3.6 mIoU and the best KPConv-based method by 4.3 mIoU.
• Replacing post-processing with a KPConv layer provides a clear accuracy gain over baseline 2D-plus-KNN approaches.
• The model leverages ResNeXt-101 features with ASPP and a 3D KPConv refinement to improve per-point labeling.
• End-to-end learnability enables the 2D projections to be effectively refined in 3D space.
• Compared to several contemporaries (RangeNet++, RandLa-Net, SqueezeSegV3, etc.), KPRNet achieves competitive or superior per-class performance and overall mIoU.