[论文解读] LiDAR Data Enrichment Using Deep Learning Based on High-Resolution Image: An Approach to Achieve High-Performance LiDAR SLAM Using Low-cost LiDAR

Yue Jiang, Weisong Wen+2 more|arXiv (Cornell University)|Aug 9, 2020

Robotics and Sensor-Based Localization参考文献 31被引用 7

一句话总结

该论文提出了一种基于深度学习的方法，利用高分辨率RGB图像增强稀疏的16通道LiDAR点云，实现低成本传感器下的高性能LiDAR SLAM。通过融合ERFNet的图像语义信息与基于稀疏卷积神经网络的密集点云预测，该方法在点云重建中实现了1.1m的均方误差，同时保持了细节更丰富的高精度SLAM地图。

ABSTRACT

LiDAR-based SLAM algorithms are extensively studied to providing robust and accurate positioning for autonomous driving vehicles (ADV) in the past decades. Satisfactory performance can be obtained using high-grade 3D LiDAR with 64 channels, which can provide dense point clouds. Unfortunately, the high price significantly prevents its extensive commercialization in ADV. The cost-effective 3D LiDAR with 16 channels is a promising replacement. However, only limited and sparse point clouds can be provided by the 16 channels LiDAR, which cannot guarantee sufficient positioning accuracy for ADV in challenging dynamic environments. The high-resolution image from the low-cost camera can provide ample information about the surroundings. However, the explicit depth information is not available from the image. Inspired by the complementariness of 3D LiDAR and camera, this paper proposes to make use of the high-resolution images from a camera to enrich the raw 3D point clouds from the low-cost 16 channels LiDAR based on a state-of-the-art deep learning algorithm. An ERFNet is firstly employed to segment the image with the aid of the raw sparse 3D point clouds. Meanwhile, the sparse convolutional neural network is employed to predict the dense point clouds based on raw sparse 3D point clouds. Then, the predicted dense point clouds are fused with the segmentation outputs from ERFnet using a novel multi-layer convolutional neural network to refine the predicted 3D point clouds. Finally, the enriched point clouds are employed to perform LiDAR SLAM based on the state-of-the-art normal distribution transform (NDT). We tested our approach on the re-edited KITTI datasets: (1)the sparse 3D point clouds are significantly enriched with a mean square error of 1.1m MSE. (2)the map generated from the LiDAR SLAM is denser which includes more details without significant accuracy loss.

研究动机与目标

• 解决低成本16通道LiDAR在自动驾驶应用中点云密度有限的挑战。
• 利用高分辨率相机图像在无需显式深度标注的情况下增强稀疏LiDAR点云。
• 通过多模态融合提升点云质量，实现使用低成本传感器的高性能LiDAR SLAM。
• 在显著提升地图密度与细节的同时，保持SLAM精度，尤其在动态环境中。

提出的方法

• 基于ERFNet的语义分割网络处理高分辨率RGB图像，并通过原始稀疏3D点云进行空间对齐。
• 基于稀疏卷积神经网络的模型从输入的稀疏LiDAR数据中预测密集3D点云。
• 提出一种新型多层卷积神经网络，将预测的密集点云与ERFNet分割输出融合，以优化几何结构与语义信息。
• 将优化后的增强点云作为输入，用于最先进的基于正态分布变换（NDT）的LiDAR SLAM系统。
• 在重新编辑的KITTI数据集上进行训练与评估，以确保方法在真实场景中的适用性。
• 通过端到端学习整合多模态数据（图像与LiDAR），以提升空间重建的保真度。

实验结果

研究问题

• RQ1 高分辨率RGB图像能否有效补偿16通道LiDAR点云在自动驾驶中的稀疏性？
• RQ2 深度学习在利用图像先验提升低成本LiDAR数据密度与几何精度方面，其提升程度如何？
• RQ3 语义分割与密集预测的融合是否能在不增加计算成本的前提下提升SLAM性能？
• RQ4 增强后的点云能否在显著提升地图细节与环境表征能力的同时，保持SLAM精度？

主要发现

• 所提方法在从稀疏16通道LiDAR数据重建密集点云时，实现了1.1m的均方误差（MSE）。
• 增强后的点云使得生成的SLAM地图更加密集，并显著提升了环境细节的表达能力。
• 尽管使用了低成本16通道LiDAR，SLAM性能依然稳定，精度下降可忽略。
• ERFNet语义分割与基于稀疏卷积的预测结果的融合，显著提升了重建点云的几何保真度。
• 该方法成功利用高分辨率图像在无需显式深度监督的情况下增强了LiDAR数据。