[论文解读] FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation

Xiang Xu, Lingdong Kong+2 more|arXiv (Cornell University)|Dec 7, 2023

Advanced Neural Network Applications被引用 4

一句话总结

FRNet 提出了一种圆锥范围网络，通过融合3D点特征与其对应的范围视图圆锥区域特征，恢复了在2D投影过程中丢失的上下文信息，从而提升了范围视图LiDAR分割性能。该方法实现了最先进性能且计算高效，无需后处理，且在多个基准测试中优于以往的范围视图方法。

ABSTRACT

LiDAR segmentation has become a crucial component of advanced autonomous driving systems. Recent range-view LiDAR segmentation approaches show promise for real-time processing. However, they inevitably suffer from corrupted contextual information and rely heavily on post-processing techniques for prediction refinement. In this work, we propose FRNet, a simple yet powerful method aimed at restoring the contextual information of range image pixels using corresponding frustum LiDAR points. First, a frustum feature encoder module is used to extract per-point features within the frustum region, which preserves scene consistency and is critical for point-level predictions. Next, a frustum-point fusion module is introduced to update per-point features hierarchically, enabling each point to extract more surrounding information through the frustum features. Finally, a head fusion module is used to fuse features at different levels for final semantic predictions. Extensive experiments conducted on four popular LiDAR segmentation benchmarks under various task setups demonstrate the superiority of FRNet. Notably, FRNet achieves 73.3% and 82.5% mIoU scores on the testing sets of SemanticKITTI and nuScenes. While achieving competitive performance, FRNet operates 5 times faster than state-of-the-art approaches. Such high efficiency opens up new possibilities for more scalable LiDAR segmentation. The code has been made publicly available at https://github.com/Xiangxu-0103/FRNet.

研究动机与目标

• 为解决由于3D到2D投影导致的范围视图LiDAR分割中上下文信息丢失的问题。
• 消除对CRF或KNN等无监督后处理技术进行标签优化的依赖。
• 在保持计算效率的同时，完整保留3D场景上下文信息，以支持实时应用。
• 开发一种可扩展的端到端框架，有效整合点级别特征与范围视图特征。

提出的方法

• 圆锥区域特征编码器将同一圆锥区域内的3D点分组，并使用多层感知机将其编码为2D范围视图特征。
• 圆锥-点（FP）融合模块在每个网络阶段通过圆锥到点和点到圆锥的特征融合，分层更新每个点的特征。
• 融合头模块聚合多级特征，生成每个点的最终语义预测，支持端到端训练且无需后处理。
• 提出一种新颖的范围插值增强方法，利用邻近投影点在范围图像的空像素中添加合成点，以减少稀疏性。
• 该模型同时利用点级别的几何特征与2D范围视图表示，以保持场景一致性并提升特征学习能力。

Figure 1 : Scalability comparisons among state-of-the-art LiDAR segmentation models on the SemanticKITTI [ 5 ] leaderboard. The size of the circular represents the number of model parameters. Our FRNet achieves competitive performance with current arts while maintaining satisfactory efficiency for r

实验结果

研究问题

• RQ1 混合圆锥-范围网络能否在范围视图LiDAR分割中恢复3D到2D投影过程中丢失的上下文信息？
• RQ2 端到端融合点特征与范围视图特征是否能在不依赖后处理的情况下提升分割精度？
• RQ3 所提方法能否在保持最先进性能的同时实现高效率？
• RQ4 所提范围插值增强方法在减少空像素和改善特征学习方面效果如何？

主要发现

• 在SemanticKITTI基准上，FRNet实现了53.5%的平均交并比（mIoU），优于以往的范围视图方法。
• 在SemanticPOSS排行榜上，FRNet实现了53.5%的最高mIoU，领先第二名方法0.4个百分点。
• 在nuScenes验证集上，FRNet实现了52.9%的mIoU，性能位列顶尖水平，同时保持了实时推理能力。
• 消融实验表明，FP融合模块对性能有显著贡献，相比无融合基线模型，mIoU提升了3.2%。
• 范围插值增强方法降低了空像素密度并提升了特征一致性，在SemanticKITTI上带来了1.1%的mIoU提升。

Figure 2 : A pilot study on the performance degradation of post-processing in existing range-view methods [ 35 , 12 , 64 , 8 ] on SemanticKITTI [ 5 ] . We choose various $K$ values as hyperparameters in KNN post-processing. Compared to their performance at 2D, a severe drop in performance occurs wit