[Paper Review] Online Extrinsic Camera Calibration for Temporally Consistent IPM Using Lane Boundary Observations with a Lane Width Prior

Jeong-Kyun Lee, Young Ki Baik+2 more|arXiv (Cornell University)|Aug 9, 2020

Advanced Vision and Imaging23 references4 citations

TL;DR

This paper proposes an online extrinsic camera calibration method that estimates pitch, yaw, roll, and camera height using lane boundary observations and a lane width prior, enabling temporally consistent inverse perspective mapping (IPM) for autonomous driving. By combining vanishing point estimation for pitch/yaw and optimization with a lane width prior for roll/camera height via extended Kalman filtering, the method achieves sub-degree accuracy and centimeter-level stability even under noise and motion variation.

ABSTRACT

In this paper, we propose a method for online extrinsic camera calibration, i.e., estimating pitch, yaw, roll angles and camera height from road surface in sequential driving scene images. The proposed method estimates the extrinsic camera parameters in two steps: 1) pitch and yaw angles are estimated simultaneously using a vanishing point computed from a set of lane boundary observations, and then 2) roll angle and camera height are computed by minimizing difference between lane width observations and a lane width prior. The extrinsic camera parameters are sequentially updated using extended Kalman filtering (EKF) and are finally used to generate a temporally consistent bird-eye-view (BEV) image by inverse perspective mapping (IPM). We demonstrate the superiority of the proposed method in synthetic and real-world datasets.

Motivation & Objective

• Address the challenge of drifting extrinsic camera parameters during driving due to vibrations and environmental changes.
• Enable real-time, automatic calibration without manual intervention, crucial for safety in ADAS and autonomous driving systems.
• Achieve temporally consistent bird's-eye-view (BEV) images by estimating all four extrinsic parameters (pitch, yaw, roll, height) online.
• Leverage lane width as a prior to improve estimation accuracy and robustness in dynamic driving conditions.
• Ensure temporal consistency in IPM outputs to prevent fluctuating lane widths and scale distortions in BEV images.

Proposed method

• Detect lane boundaries in input images using a fully convolutional network-based segmentation model.
• Estimate the vanishing point (VP) of parallel lane boundaries using RANSAC and Gaussian sphere theory for robustness to noise.
• Compute pitch and yaw angles from the VP direction relative to the camera's forward vector.
• Use the estimated pitch and yaw to compute roll angle and camera height by minimizing the difference between observed and prior lane widths.
• Apply extended Kalman filtering (EKF) to sequentially update the extrinsic parameters, ensuring temporal consistency.
• Generate temporally stable BEV images via inverse perspective mapping (IPM) using the updated parameters.

Experimental results

Research questions

• RQ1 Can all four extrinsic camera parameters (pitch, yaw, roll, camera height) be estimated online using only lane boundary observations?
• RQ2 How can a lane width prior be effectively used to estimate roll and camera height when pitch and yaw are known?
• RQ3 Can the proposed method produce temporally consistent BEV images despite camera motion and image noise?
• RQ4 How accurate is the parameter estimation under varying levels of image noise and real-world driving conditions?
• RQ5 To what extent does the use of EKF improve the temporal stability of the calibration results?

Key findings

• The method achieves root mean square error (RMSE) of less than 0.2 degrees for pitch and yaw, and less than 1.5 cm for camera height, even under high noise (σ² = 9 pixels).
• In synthetic datasets, the RMSE for roll angle remains below 0.114 degrees and for height below 1.4 cm at σ² = 9, demonstrating robustness to noise.
• BEV images generated by the proposed method show consistent lane widths and minimal fluctuation across frames, even with motion variation.
• In real-world datasets (Test1 and Test2), the method significantly improves BEV quality compared to baseline IPM using pre-calibrated parameters, eliminating scale inconsistencies.
• The estimated extrinsic parameters closely track the pseudo-ground truth derived from manually annotated lane boundaries, validating accuracy on real data.
• The use of EKF ensures smooth, temporally coherent parameter updates, preventing abrupt jumps in the BEV output.