[Paper Review] SensorX2car: Sensors-to-car calibration for autonomous driving in road scenarios

Guohang Yan, Zhaotong Luo+2 more|arXiv (Cornell University)|Jan 18, 2023

Robotics and Sensor-Based Localization4 citations

TL;DR

SensorX2car introduces the first open-source toolbox for online, target-free sensor-to-car calibration in autonomous driving, enabling precise calibration of camera, LiDAR, GNSS/INS, and millimeter-wave radar to the vehicle body frame using sensor-specific patterns—image features, 3D points, pose solutions, and radar speed—achieving sub-degree accuracy within minutes of normal driving.

ABSTRACT

Properly-calibrated sensors are the prerequisite for a dependable autonomous driving system. However, most prior methods focus on extrinsic calibration between sensors, and few focus on the misalignment between the sensors and the vehicle coordinate system. Existing targetless approaches rely on specific prior knowledge, such as driving routes and road features, to handle this misalignment. This work removes these limitations and proposes more general calibration methods for four commonly used sensors: Camera, LiDAR, GNSS/INS, and millimeter-wave Radar. By utilizing sensor-specific patterns: image feature, 3D LiDAR points, GNSS/INS solved pose, and radar speed, we design four corresponding methods to mainly calibrate the rotation from sensor to car during normal driving within minutes, composing a toolbox named SensorX2car. Real-world and simulated experiments demonstrate the practicality of our proposed methods. Meanwhile, the related codes have been open-sourced to benefit the community. To the best of our knowledge, SensorX2car is the first open-source sensor-to-car calibration toolbox. The code is available at https://github.com/OpenCalib/SensorX2car.

Motivation & Objective

• Address the lack of general-purpose, online calibration methods for sensor-to-car extrinsic parameters, especially rotation, in real-world driving scenarios.
• Overcome limitations of prior methods that rely on specific road features, lane markings, or controlled environments.
• Develop a practical, accessible calibration solution that works across diverse driving routes without requiring special markers or facilities.
• Enable reliable, repeatable calibration of four key sensors—camera, LiDAR, GNSS/INS, and radar—using only data collected during normal operation.
• Provide an open-source toolbox to support community adoption and further research in autonomous vehicle sensor calibration.

Proposed method

• For camera calibration, use a deep learning network to predict vanishing points and horizon lines from single images, converting them into roll, pitch, and yaw angles relative to the vehicle frame.
• For LiDAR, estimate the 6-DoF pose via SLAM and calibrate rotation by averaging the yaw difference between vehicle forward direction and LiDAR forward direction, with pitch and roll derived from ground plane extraction.
• For GNSS/INS, use the integrated 6-DoF pose to estimate the yaw angle relative to the vehicle, assuming vehicle trajectory aligns with heading direction.
• For radar, calibrate yaw by fitting a cosine curve to the relationship between measured object speed and vehicle speed, improving robustness by incorporating position information to reduce noise.
• Accumulate data over time and filter outliers to improve consistency, especially for roll and pitch angles affected by turning dynamics.
• Apply trajectory-based assumptions to estimate vehicle-to-world orientation, enabling conversion of sensor-to-world rotation to sensor-to-car rotation.

Experimental results

Research questions

• RQ1 Can sensor-to-car calibration be achieved without relying on specific road features like lane lines or artificial markers?
• RQ2 Can online calibration methods for camera, LiDAR, GNSS/INS, and radar be designed using only intrinsic sensor patterns and normal driving data?
• RQ3 How accurately can rotation parameters (yaw, pitch, roll) be estimated from real-world driving data using sensor-specific signal characteristics?
• RQ4 What is the consistency and robustness of the calibration results across diverse driving conditions and routes?
• RQ5 Can a unified, open-source toolbox be built to support sensor-to-car calibration across multiple sensor types in real-world autonomous driving systems?

Key findings

• The camera calibration method achieved sub-degree accuracy in roll, pitch, and yaw estimation using a deep learning model trained on a KITTI-based dataset, with standard deviation of 0.1° for yaw and pitch, and 0.5° for roll.
• LiDAR calibration achieved a yaw and pitch standard deviation of around 0.1° across all driving segments, dropping to 0.1° for straight-line segments, with roll standard deviation of 0.5° due to vehicle tilt during turns.
• GNSS/INS calibration showed a yaw fluctuation range of approximately 0.1° to 0.2° across sequences, with consistent trend matching between vehicle and IMU yaw angles.
• Radar calibration using simulation data achieved absolute error below 0.1°, while real-world data improved from high noise to a fluctuation range of 0.3–0.4° in straight-line segments after incorporating position information.
• The overall calibration consistency across four data periods was validated through standard deviation and distribution analysis, confirming robustness and reliability.
• The open-source toolbox SensorX2car, available at https://github.com/OpenCalib/SensorX2car, enables practical, general-purpose calibration across diverse driving scenarios without route or feature constraints.