[Paper Review] Real Time Lidar and Radar High-Level Fusion for Obstacle Detection and Tracking with evaluation on a ground truth

Hatem Hajri, Mohamed-Cherif Rahal|arXiv (Cornell University)|Jul 30, 2018

Autonomous Vehicle Technology and Safety18 citations

TL;DR

This paper presents a real-time, high-level fusion algorithm combining Lidar and Radar data using a Global Nearest Neighbor (GNN) filter based on Kalman filtering for obstacle detection and tracking in autonomous vehicles. Evaluated against RTK-based ground truth, the fusion method reduces mean square error (MSE) in relative position and velocity estimates compared to individual sensors, demonstrating improved accuracy and robustness, especially during sensor dropouts or in challenging driving scenarios like high-speed highway and curved road tracking.

ABSTRACT

- Both Lidars and Radars are sensors for obstacle detection. While Lidars are very accurate on obstacles positions and less accurate on their velocities, Radars are more precise on obstacles velocities and less precise on their positions. Sensor fusion between Lidar and Radar aims at improving obstacle detection using advantages of the two sensors. The present paper proposes a real-time Lidar/Radar data fusion algorithm for obstacle detection and tracking based on the global nearest neighbour standard filter (GNN). This algorithm is implemented and embedded in an automative vehicle as a component generated by a real-time multisensor software. The benefits of data fusion comparing with the use of a single sensor are illustrated through several tracking scenarios (on a highway and on a bend) and using real-time kinematic sensors mounted on the ego and tracked vehicles as a ground truth.

Motivation & Objective

• Address the lack of quantitative, real-world ground truth comparisons between Lidar, Radar, and their fusion in autonomous driving scenarios.
• Develop a real-time, high-level fusion algorithm that combines Lidar and Radar data to improve obstacle detection and tracking performance.
• Evaluate the fusion system's accuracy and robustness against sensor unavailability using precise RTK sensors as ground truth.
• Demonstrate that fusion provides a balanced compromise between the strengths of Lidar (position accuracy) and Radar (velocity accuracy) in dynamic driving conditions.

Proposed method

• Implement a high-level fusion framework using the Global Nearest Neighbor (GNN) filter, which integrates object detections from Lidar and Radar at the track level.
• Apply a Kalman filter-based estimation process to predict and update object states (position, velocity) using measurements from both sensors.
• Use a centralized fusion architecture where five Ibeo LUX Lidars (25 Hz) and one Ibeo ARS 308 Radar (15 Hz) feed data to a central Fusion Box for processing.
• Incorporate CAN bus odometry and RTK sensor data (0.02 m position, 0.02 m/s velocity precision) to generate ground truth for performance evaluation.
• Synchronize data from two autonomous vehicles equipped with RTK sensors to enable accurate relative state estimation during car-following experiments.
• Apply a real-time implementation with processing latency under 15 microseconds, ensuring suitability for embedded automotive systems.

Experimental results

Research questions

• RQ1 How does the fusion of Lidar and Radar data compare to individual sensor performance in terms of mean square error (MSE) for relative position and velocity estimation?
• RQ2 To what extent does the fusion algorithm improve tracking robustness during periods of sensor unavailability or signal dropout?
• RQ3 How does the fusion output compare to RTK-based ground truth in real-world highway and curved-road driving scenarios?
• RQ4 Can the fusion method achieve a balanced trade-off between the high positional accuracy of Lidar and the superior velocity estimation of Radar?
• RQ5 What is the temporal smoothness and reliability of the fused state estimates compared to raw Lidar and Radar measurements?

Key findings

• On the highway, the fusion method achieved an MSE of 0.39 m for relative x-position, outperforming Lidar (0.55 m) and approaching Radar’s 0.33 m, while also reducing y-position MSE to 0.21 m compared to Lidar’s 0.16 m and Radar’s 0.43 m.
• For velocity estimation on the highway, fusion reduced MSE in vx to 0.19 m/s (vs. 0.28 m/s for Lidar and 0.15 m/s for Radar), and in vy to 0.27 m/s (vs. 0.31 m/s for Lidar and 0.25 m/s for Radar), showing a balanced improvement.
• On curved roads, fusion reduced x-position MSE to 0.65 m (vs. 0.8 m for Radar and 0.61 m for Lidar), and y-position MSE to 0.19 m (vs. 0.5 m for Radar and 0.15 m for Lidar), demonstrating consistent improvement in lateral accuracy.
• The fusion output exhibited significantly smoother temporal trajectories for vx and vy compared to Lidar, which showed abrupt jumps and piecewise constancy, while Radar exhibited multiple detection dropouts.
• Fusion demonstrated superior robustness: it maintained stable tracking during Lidar and Radar unavailability periods, unlike individual sensors, which were prone to complete loss of track.
• Radar outperformed Lidar in x-position estimation on the highway (0.33 m vs. 0.55 m MSE), a finding corroborated by prior work, highlighting the importance of sensor-specific characteristics in fusion design.