[Paper Review] CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes

Tao Ma, Zhizheng Liu+2 more|arXiv (Cornell University)|Mar 8, 2021

Robotics and Sensor-Based Localization21 references28 citations

TL;DR

CRLF introduces a fully automatic LiDAR-Camera extrinsic calibration method using line features from road lanes and poles, providing a coarse P3L-based initialization followed by refinement with a semantic line-based cost function.

ABSTRACT

For autonomous vehicles, an accurate calibration for LiDAR and camera is a prerequisite for multi-sensor perception systems. However, existing calibration techniques require either a complicated setting with various calibration targets, or an initial calibration provided beforehand, which greatly impedes their applicability in large-scale autonomous vehicle deployment. To tackle these issues, we propose a novel method to calibrate the extrinsic parameter for LiDAR and camera in road scenes. Our method introduces line features from static straight-line-shaped objects such as road lanes and poles in both image and point cloud and formulates the initial calibration of extrinsic parameters as a perspective-3-lines (P3L) problem. Subsequently, a cost function defined under the semantic constraints of the line features is designed to perform refinement on the solved coarse calibration. The whole procedure is fully automatic and user-friendly without the need to adjust environment settings or provide an initial calibration. We conduct extensive experiments on KITTI and our in-house dataset, quantitative and qualitative results demonstrate the robustness and accuracy of our method.

Motivation & Objective

• Automate LiDAR-camera extrinsic calibration without calibration targets or prior initialization.
• Leverage static straight-line road-world features (lanes and poles) as calibration anchors.
• Provide a robust initial calibration via perspective-3-lines (P3L) and refine using semantic line constraints.
• Demonstrate effectiveness and efficiency on KITTI and in-house datasets in realistic road scenarios.

Proposed method

• Extract line features from both image and LiDAR data focusing on lanes and poles.
• Formulate coarse calibration by solving a P3L problem using line correspondences.
• Compute a cost function under semantic line constraints to refine calibration by matching line masks after projection.
• Refine the extrinsics by optimizing the cost function with a stochastic search around the coarse solution.
• Use a ground-parallel intermediate step to simplify initial P3L solving and then remove the parallel assumption in refinement.

Experimental results

Research questions

• RQ1 How can LiDAR-Camera extrinsic calibration be achieved automatically in road scenes without calibration targets or prior initialization?
• RQ2 Can line features from static road objects (lanes and poles) provide sufficient constraints for accurate coarse calibration and robust refinement?
• RQ3 What is the effectiveness of a P3L-based initialization followed by semantic-line refinement on real datasets?
• RQ4 How does CRLF perform compared with reference calibrations on KITTI and in-house data in terms of translation and rotation accuracy?

Key findings

• CRLF achieves automatic LiDAR-Camera extrinsic calibration using line features within road scenes.
• A coarse calibration from line correspondences is obtained by solving the P3L problem and transformed via a ground-parallel intermediate frame.
• The refinement leveraging a semantic line-based cost function significantly improves accuracy over the coarse result.
• On KITTI, coarse calibration yields translation errors around 0.121–0.182 m and rotation errors around 0.628–1.043 degrees; refinement reduces these errors substantially.
• On in-house data, coarse calibration yields translation errors around 0.053–0.074 m and rotation errors around 0.684–1.092 degrees; refinement reduces these to 0.018–0.015 m and 0.332–0.613 degrees respectively.
• The end-to-end CRLF process runs in about 0.3 seconds per frame, demonstrating efficiency for large-scale deployment.