[Paper Review] Learning to Localize Using a LiDAR Intensity Map

Ioan Andrei Bârsan, Shenlong Wang+2 more|arXiv (Cornell University)|Dec 20, 2020

Robotics and Sensor-Based Localization64 citations

TL;DR

The paper presents a real-time, calibration-agnostic localization system that embeds online LiDAR sweeps and aLiDAR intensity map into a shared deep space and localizes via efficient convolutional matching, achieving centimeter-level accuracy at 15 Hz across sensors.

ABSTRACT

In this paper we propose a real-time, calibration-agnostic and effective localization system for self-driving cars. Our method learns to embed the online LiDAR sweeps and intensity map into a joint deep embedding space. Localization is then conducted through an efficient convolutional matching between the embeddings. Our full system can operate in real-time at 15Hz while achieving centimeter level accuracy across different LiDAR sensors and environments. Our experiments illustrate the performance of the proposed approach over a large-scale dataset consisting of over 4000km of driving.

Motivation & Objective

• Motivate centimeter-level vehicle localization for HD-map-based perception and planning in autonomous driving.
• Propose a calibration-agnostic localization framework that operates across different LiDAR sensors.
• Develop a deep embedding approach for online LiDAR sweeps and pre-built LiDAR intensity maps.
• Enable real-time localization through efficient frequency-domain convolutional matching.

Proposed method

• Embed online LiDAR BEV intensity images and pre-built intensity maps into a common neural embedding space.
• Compute LiDAR pose likelihoods by cross-correlating rotated online embedding with the map embedding in the Fourier domain.
• Model localization as a deep recursive Bayesian update combining LiDAR, GPS, and motion priors.
• Use a histogram-filter-like discrete search over a 3-DoF pose (x, y, theta) centered at the dead-reckoning pose.
• Train the system end-to-end with a cross-entropy loss on the resulting pose score map.
• Adopt a soft argmax for smoother pose estimates and robustness to observation noise.

Experimental results

Research questions

• RQ1 Can a learned embedding space enable calibration-free LiDAR-based localization across different sensors?
• RQ2 What is the accuracy and robustness of the proposed embedding-based localization under real-time constraints?
• RQ3 How well does the system generalize from one LiDAR modality to another and across urban/highway environments?
• RQ4 What is the impact of using velocity/motion priors and probabilistic inference on localization robustness?

Key findings

• Achieves real-time localization at 15 Hz with centimeter-level accuracy on diverse highway and urban scenes.
• Outperforms ICP and raw LiDAR matching baselines in median error and especially in worst-case (failure rate) scenarios.
• Demonstrates cross-sensor/generalization capability, maintaining accuracy when transferring between LiDAR A and LiDAR B datasets.
• FFT-based convolution dramatically speeds up matching, enabling efficient search over rotation and translation in a 3-DoF space.
• Incorporating motion priors and probabilistic inference improves robustness and reduces failure rates.
• Single-channel embeddings with LinkNet backbones provide a favorable balance of accuracy and runtime.