[Paper Review] SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving

Tiago Cortinhal, George Tzelepis+1 more|arXiv (Cornell University)|Mar 7, 2020

Advanced Neural Network ApplicationsComputer Science47 references100 citations

TL;DR

SalsaNext extends SalsaNet with a context module, dilated residual blocks, pixel-shuffle upsampling, Lovász-Softmax loss, and Bayesian uncertainty estimation to achieve real-time, uncertainty-aware LiDAR semantic segmentation, ranking first on Semantic-KITTI.

ABSTRACT

In this paper, we introduce SalsaNext for the uncertainty-aware semantic segmentation of a full 3D LiDAR point cloud in real-time. SalsaNext is the next version of SalsaNet [1] which has an encoder-decoder architecture where the encoder unit has a set of ResNet blocks and the decoder part combines upsampled features from the residual blocks. In contrast to SalsaNet, we introduce a new context module, replace the ResNet encoder blocks with a new residual dilated convolution stack with gradually increasing receptive fields and add the pixel-shuffle layer in the decoder. Additionally, we switch from stride convolution to average pooling and also apply central dropout treatment. To directly optimize the Jaccard index, we further combine the weighted cross-entropy loss with Lovasz-Softmax loss [2]. We finally inject a Bayesian treatment to compute the epistemic and aleatoric uncertainties for each point in the cloud. We provide a thorough quantitative evaluation on the Semantic-KITTI dataset [3], which demonstrates that the proposed SalsaNext outperforms other state-of-the-art semantic segmentation networks and ranks first on the Semantic-KITTI leaderboard. We also release our source code https://github.com/TiagoCortinhal/SalsaNext.

Motivation & Objective

• Motivate reliable, real-time semantic understanding for autonomous driving using 3D LiDAR data.
• Develop a network that yields accurate per-point labels with quantified epistemic and aleatoric uncertainties.
• Improve contextual awareness and efficiency over previous SalsaNet while maintaining real-time performance.

Proposed method

• Introduce a context module with a residual dilated convolution stack to capture global 360-degree context.
• Replace encoder ResNet blocks with a stack of dilated convolutions (rates 2, kernels 3/5/7) with concatenation and residual connections.
• Use a pixel-shuffle layer in the decoder to avoid checkerboard artifacts and improve upsampling efficiency.
• Apply central encoder-decoder dropout, switch encoder downsampling to average pooling, and use a 1x1 conv to match class channels.
• Optimize training using a loss that combines weighted cross entropy with Lovász-Softmax loss to directly maximize IoU.
• Incorporate Bayesian treatment to estimate both epistemic and aleatoric uncertainties without retraining, via MC sampling and ADF-inspired propagation.

Experimental results

Research questions

• RQ1 Can SalsaNext outperform state-of-the-art projection-based and point-wise 3D LiDAR segmentation methods on Semantic-KITTI?
• RQ2 Does the model provide reliable uncertainty estimates (epistemic and aleatoric) for LiDAR segmentation?
• RQ3 What is the impact of architectural changes (context module, dilated blocks, pixel-shuffle, loss function) on segmentation accuracy and efficiency?

Key findings

• SalsaNext achieves 59.5% mean IoU on Semantic-KITTI test set, outperforming prior methods and ranking first on the leaderboard.
• Ablation shows cumulative gains from context module, dilated convolutions, pixel-shuffle, and Lovász-Softmax loss, with a final gain of about 0.7 percentage points on mean IoU over SalsaNet.
• SalsaNext provides competitive runtime (approximately 41.26 ms total per frame) with a modest parameter increase (about 0.15M) over SalsaNet.
• The model delivers both epistemic and aleatoric uncertainties per point, enabling uncertainty-aware predictions for safer autonomy.
• Qualitative results illustrate lower uncertainty on confident predictions and higher uncertainty at boundaries or distant objects.
• SalsaNext achieves higher mean IoU (59.5) than SalsaNet variants and yields better per-class performance in several categories (e.g., road, vegetation, terrain).
• Runtime comparisons show SalsaNext strikes a balance between accuracy and speed, achieving 24 Hz throughput with 6.73M parameters and 125.68 GFLOPs.