[Paper Review] Joint Calibration of Panoramic Camera and Lidar Based on Supervised Learning

Mingwei Cao, Ming Quan Yang+3 more|arXiv (Cornell University)|Sep 9, 2017

Advanced Vision and Imaging7 references3 citations

TL;DR

This paper proposes a supervised learning-based method for joint calibration of panoramic cameras and LiDAR sensors, using a round calibration target to extract corresponding feature points and train a neural network to regress rotation and translation matrices. The approach achieves higher accuracy and automation than traditional optimization-based methods, with experimental results showing superior performance on real-world data.

ABSTRACT

In view of contemporary panoramic camera-laser scanner system, the traditional calibration method is not suitable for panoramic cameras whose imaging model is extremely nonlinear. The method based on statistical optimization has the disadvantage that the requirement of the number of laser scanner's channels is relatively high. Calibration equipments with extreme accuracy for panoramic camera-laser scanner system are costly. Facing all these in the calibration of panoramic camera-laser scanner system, a method based on supervised learning is proposed. Firstly, corresponding feature points of panoramic images and point clouds are gained to generate the training dataset by designing a round calibration object. Furthermore, the traditional calibration problem is transformed into a multiple nonlinear regression optimization problem by designing a supervised learning network with preprocessing of the panoramic imaging model. Back propagation algorithm is utilized to regress the rotation and translation matrix with high accuracy. Experimental results show that this method can quickly regress the calibration parameters and the accuracy is better than the traditional calibration method and the method based on statistical optimization. The calibration accuracy of this method is really high, and it is more highly-automated.

Motivation & Objective

• To address the limitations of traditional calibration methods for panoramic cameras due to their highly nonlinear imaging model.
• To overcome the high channel count requirement and cost of precision calibration equipment in panoramic camera-LiDAR systems.
• To develop a more automated and accurate calibration method using supervised learning instead of iterative optimization.
• To transform the calibration problem into a multiple nonlinear regression task via a tailored neural network architecture.

Proposed method

• A round calibration target is designed to generate corresponding feature points between panoramic images and LiDAR point clouds.
• The panoramic imaging model is preprocessed to enable effective feature matching and network input preparation.
• A supervised learning network is trained using the generated dataset to regress the 3D transformation (rotation and translation) between the camera and LiDAR.
• The network employs backpropagation to optimize the regression of calibration parameters from image and point cloud features.
• The method treats the calibration problem as a nonlinear regression task, avoiding reliance on iterative optimization or high-channel LiDAR systems.

Experimental results

Research questions

• RQ1 Can a supervised learning approach achieve higher calibration accuracy than traditional optimization-based methods for panoramic camera-LiDAR systems?
• RQ2 Does the proposed method reduce dependency on high-channel LiDAR sensors compared to statistical optimization techniques?
• RQ3 To what extent does the use of a round calibration target improve feature correspondence and calibration robustness?
• RQ4 How does the proposed method compare in automation and convergence speed to conventional calibration pipelines?

Key findings

• The proposed method achieves higher calibration accuracy than both traditional calibration and statistical optimization-based approaches.
• The method demonstrates faster convergence and higher automation due to the end-to-end learning framework.
• Experimental results confirm that the supervised learning network effectively regresses rotation and translation matrices with high precision.
• The approach is robust to the nonlinear imaging model of panoramic cameras, which traditional methods struggle to handle.