[论文解读] Sense Less, Generate More: Pre-training LiDAR Perception with Masked Autoencoders for Ultra-Efficient 3D Sensing

Sina Tayebati, Theja Tulabandhula+1 more|arXiv (Cornell University)|Jun 12, 2024

3D Surveying and Cultural Heritage被引用 4

一句话总结

本文提出径向掩码自编码（R-MAE），一种用于激光雷达感知的生成式预训练方法，通过掩码径向角度区域并利用掩码自编码器生成缺失部分，从而降低感知能耗。R-MAE在Waymo数据集上实现高达5.59%的mAP/mAPH提升（90%掩码率），在KITTI数据集上将小目标检测性能提升4.37%，展示了兼具高效率与高精度的3D感知能力。

ABSTRACT

In this work, we propose a disruptively frugal LiDAR perception dataflow that generates rather than senses parts of the environment that are either predictable based on the extensive training of the environment or have limited consequence to the overall prediction accuracy. Therefore, the proposed methodology trades off sensing energy with training data for low-power robotics and autonomous navigation to operate frugally with sensors, extending their lifetime on a single battery charge. Our proposed generative pre-training strategy for this purpose, called as radially masked autoencoding (R-MAE), can also be readily implemented in a typical LiDAR system by selectively activating and controlling the laser power for randomly generated angular regions during on-field operations. Our extensive evaluations show that pre-training with R-MAE enables focusing on the radial segments of the data, thereby capturing spatial relationships and distances between objects more effectively than conventional procedures. Therefore, the proposed methodology not only reduces sensing energy but also improves prediction accuracy. For example, our extensive evaluations on Waymo, nuScenes, and KITTI datasets show that the approach achieves over a 5% average precision improvement in detection tasks across datasets and over a 4% accuracy improvement in transferring domains from Waymo and nuScenes to KITTI. In 3D object detection, it enhances small object detection by up to 4.37% in AP at moderate difficulty levels in the KITTI dataset. Even with 90% radial masking, it surpasses baseline models by up to 5.59% in mAP/mAPH across all object classes in the Waymo dataset. Additionally, our method achieves up to 3.17% and 2.31% improvements in mAP and NDS, respectively, on the nuScenes dataset, demonstrating its effectiveness with both single and fused LiDAR-camera modalities. https://github.com/sinatayebati/Radial_MAE.

研究动机与目标

• 为解决主动激光雷达传感器在低功耗机器人与自动驾驶系统中能耗过高的问题。
• 通过生成可预测或影响较小的环境区域，而非实际测量，以降低感知能耗。
• 通过新型径向掩码策略进行生成式预训练，提升3D目标检测精度。
• 通过最小化激光激活次数，实现超省电的激光雷达运行，同时保持或提升性能。
• 在Waymo、nuScenes和KITTI等不同领域之间实现鲁棒且可泛化的表征迁移。

提出的方法

• R-MAE采用感知距离的径向掩码策略，在实际感知过程中选择性禁用特定角度区域的激光发射。
• 该方法使用3D空间稀疏卷积编码器，从激光雷达点云中未掩码的体素中提取特征。
• 通过二分类任务预测掩码区域内的体素占据情况，实现对完整3D场景的重建，即掩码自编码器。
• 通过控制掩码比例和角度扇区大小，探索感知能耗与模型性能之间的权衡。
• 模型在大规模数据集（Waymo、nuScenes、KITTI）上进行预训练，并在下游检测任务上进行微调。
• 该方法利用激光功率与R⁴成正比的特性，结合生成式重建技术，使低功耗感知更加高效。

Figure 1 : (a) Radially masked autoencoding (R-MAE) strategy: Angular regions are completely masked out as shown in the black by turning off laser emissions. Even on the sensed regions, points are probabilistically dropped-off proportional to their distance $R$ . Notably, $P_{\text{laser}}\sim R^{4}

实验结果

研究问题

• RQ1 使用径向掩码进行生成式预训练是否能在不降低检测性能的前提下减少激光雷达感知能耗？
• RQ2 与随机块掩码相比，径向掩码在保留3D场景中空间关系和物体距离方面表现如何？
• RQ3 在极端掩码比例（如90%）下，R-MAE在多大程度上能保持或提升检测精度？
• RQ4 当将来自Waymo或nuScenes的表征迁移到KITTI时，R-MAE是否能提升域泛化能力？
• RQ5 R-MAE性能对径向掩码策略中掩码扇区角度大小的敏感程度如何？

主要发现

• R-MAE在Waymo、nuScenes和KITTI数据集上均实现了超过5%的平均精度提升。
• 在KITTI数据集上，R-MAE在中等难度水平下将小目标检测的AP提升最高达4.37%。
• 在90%径向掩码率下，R-MAE在Waymo数据集上的mAP/mAPH相比基线模型最高提升5.59%。
• 在nuScenes数据集上，R-MAE在单模态和融合激光雷达-相机模态下，mAP最高提升3.17%，NDS最高提升2.31%。
• 在域迁移任务中，R-MAE实现超过4%的精度提升，其中从Waymo迁移到KITTI时，行人检测的增益达4.09%。
• 即使在92%的掩码率下，模型仍保持高性能，表明仅需感知激光雷达鸟瞰图（BEV）的8%即可实现有效运行。

Figure 2 : Accuracy under varying pretraining conditions, (a) at varying masking ratios with a fixed angular range of 1 degree, and (b) at different angular ranges with a fixed masking ratio of 0.8. Results are compared against the state-of-the-art (SOTA) PointPillars [ 39 ] method.