[Paper Review] MaskRange: A Mask-classification Model for Range-view based LiDAR Segmentation

Yi Gu, Yuming Huang+2 more|arXiv (Cornell University)|Jun 24, 2022

Advanced Neural Network Applications6 citations

TL;DR

MaskRange proposes a novel mask-classification paradigm for range-view LiDAR segmentation, outperforming existing per-pixel methods by leveraging a unified architecture for semantic and panoptic segmentation. It achieves state-of-the-art performance with 66.10 mIoU on SemanticKITTI semantic segmentation and 53.10 PQ on panoptic segmentation at 25 FPS, enabled by a novel data augmentation method, Weighted Paste Drop (WPD), which mitigates overfitting, context reliance, and class imbalance.

ABSTRACT

Range-view based LiDAR segmentation methods are attractive for practical applications due to their direct inheritance from efficient 2D CNN architectures. In literature, most range-view based methods follow the per-pixel classification paradigm. Recently, in the image segmentation domain, another paradigm formulates segmentation as a mask-classification problem and has achieved remarkable performance. This raises an interesting question: can the mask-classification paradigm benefit the range-view based LiDAR segmentation and achieve better performance than the counterpart per-pixel paradigm? To answer this question, we propose a unified mask-classification model, MaskRange, for the range-view based LiDAR semantic and panoptic segmentation. Along with the new paradigm, we also propose a novel data augmentation method to deal with overfitting, context-reliance, and class-imbalance problems. Extensive experiments are conducted on the SemanticKITTI benchmark. Among all published range-view based methods, our MaskRange achieves state-of-the-art performance with $66.10$ mIoU on semantic segmentation and promising results with $53.10$ PQ on panoptic segmentation with high efficiency. Our code will be released.

Motivation & Objective

• To investigate whether the mask-classification paradigm—successful in image segmentation—can improve range-view LiDAR segmentation performance.
• To address challenges such as overfitting, context reliance, and class imbalance in LiDAR segmentation using a novel data augmentation strategy.
• To develop a unified framework that supports both semantic and panoptic segmentation without additional supervision or hyperparameter tuning.
• To demonstrate the generalizability of the mask-classification paradigm across different LiDAR data representations, using range-view as a primary example.

Proposed method

• Adapts the MaskFormer meta-architecture to LiDAR range-view data, formulating segmentation as a set of mask-classification predictions rather than per-pixel classification.
• Introduces Weighted Paste Drop (WPD), a novel data augmentation method that pastes patches from different scans with class- and context-aware weighting to reduce overfitting and class imbalance.
• Employs a unified weighted focal loss to re-balance training samples, improving optimization stability and performance on long-tail classes.
• Uses deep supervision with improved training dynamics (DeepB) and data-dependent upsampling (DU) to enhance feature refinement and prediction accuracy.
• Applies a transformer decoder to aggregate global context, reducing reliance on local context aggregation in the backbone.
• Optimizes inference efficiency by leveraging 2D CNN efficiency and maintaining low FLOPs (2.2 GFlops extra) and latency (40 ms, 25 FPS) on a single RTX 2080Ti.

Experimental results

Research questions

• RQ1 Can the mask-classification paradigm, successful in image segmentation, be effectively transferred to range-view LiDAR segmentation to outperform per-pixel classification?
• RQ2 How can overfitting, context reliance, and class imbalance—common in sparse LiDAR data—be mitigated in a mask-classification framework?
• RQ3 Can a unified model architecture achieve high performance on both semantic and panoptic segmentation tasks in LiDAR without task-specific head tuning or bounding-box supervision?
• RQ4 To what extent does a novel data augmentation strategy like WPD improve generalization and performance in sparse, long-tailed LiDAR segmentation?

Key findings

• MaskRange achieves state-of-the-art performance with 66.10 mIoU on the SemanticKITTI semantic segmentation benchmark, ranking first among all published range-view based methods.
• It achieves 53.10 PQ on the panoptic segmentation benchmark, demonstrating strong generalization and robustness without additional supervision.
• The proposed Weighted Paste Drop (WPD) augmentation improves mIoU by 7.70% over a baseline when applied to MaskRange, significantly outperforming Mix3D and Instance Paste.
• The model runs at 25 FPS (40 ms inference time) on a single NVIDIA RTX 2080Ti, demonstrating high efficiency suitable for real-time applications.
• Ablation studies confirm that the combination of WPD, weighted focal loss, and improved deep supervision (DeepB) is essential for performance, with WPD contributing the largest gain.
• The model's performance is more sensitive to WPD than to per-pixel baselines, indicating that the mask-classification architecture better exploits the augmentation's benefits.