[Paper Review] Degeneracy in Self-Calibration Revisited and a Deep Learning Solution for Uncalibrated SLAM

Bingbing Zhuang, Quoc-Huy Tran+4 more|arXiv (Cornell University)|Jul 30, 2019

Robotics and Sensor-Based Localization44 references4 citations

TL;DR

This paper revisits the geometric degeneracy in two-view self-calibration of radial distortion and proposes a deep learning-based single-image camera self-calibration method using a CNN with deep supervision. Trained on synthetic data, the model enables robust, checkerboard-free SLAM on uncalibrated videos from KITTI Raw and YouTube, achieving near-ground-truth trajectory accuracy with marginal error gaps in quantitative evaluation.

ABSTRACT

Self-calibration of camera intrinsics and radial distortion has a long history of research in the computer vision community. However, it remains rare to see real applications of such techniques to modern Simultaneous Localization And Mapping (SLAM) systems, especially in driving scenarios. In this paper, we revisit the geometric approach to this problem, and provide a theoretical proof that explicitly shows the ambiguity between radial distortion and scene depth when two-view geometry is used to self-calibrate the radial distortion. In view of such geometric degeneracy, we propose a learning approach that trains a convolutional neural network (CNN) on a large amount of synthetic data. We demonstrate the utility of our proposed method by applying it as a checkerboard-free calibration tool for SLAM, achieving comparable or superior performance to previous learning and hand-crafted methods.

Motivation & Objective

• To theoretically prove the degeneracy in two-view self-calibration of radial distortion under forward motion using a general distortion model and discrete SfM formulation.
• To address the instability of geometric methods in real-world driving scenarios where forward motion induces ambiguity between radial distortion and scene depth.
• To develop a data-driven approach that learns camera intrinsics and radial distortion from a single uncalibrated image, enabling robust SLAM without checkerboard calibration.
• To demonstrate the feasibility and superiority of the proposed method in uncalibrated SLAM on real-world video sequences from KITTI Raw and YouTube.
• To show that deep supervision in CNNs improves performance over standard multi-task learning for intrinsic and distortion estimation.

Proposed method

• A general distortion model and discrete Structure from Motion (SfM) formulation are used to formally prove the degeneracy of two-view radial distortion self-calibration under forward motion.
• A large-scale synthetic dataset is generated by randomly varying camera intrinsics and applying random radial distortion to calibrated images, simulating diverse real-world conditions.
• A convolutional neural network (CNN) with deep supervision is trained to predict camera intrinsics (focal length, principal point) and radial distortion parameters from a single uncalibrated image.
• Deep supervision is applied to intermediate layers of the CNN to improve regularization and estimation accuracy for both radial distortion and intrinsic parameters.
• A two-step SLAM pipeline is employed: first, the network estimates camera parameters from the first 100 frames; second, these parameters are used to calibrate the entire video sequence before feeding into ORB-SLAM.
• The method is evaluated on KITTI Raw and YouTube datasets using ORB-SLAM with ground-truth parameters and hand-crafted methods as baselines.

Experimental results

Research questions

• RQ1 What is the theoretical cause of degeneracy in two-view radial distortion self-calibration under forward motion, and how can it be formally proven?
• RQ2 How does the ambiguity between radial distortion and scene depth affect the stability of geometric self-calibration methods in driving scenarios?
• RQ3 Can a deep learning model trained on synthetic data generalize to real-world uncalibrated videos and outperform traditional geometric and hand-crafted methods?
• RQ4 Does deep supervision in a CNN improve the accuracy of single-image camera self-calibration for radial distortion and intrinsic parameters?
• RQ5 Is it feasible to achieve robust SLAM on uncalibrated videos without checkerboard calibration using the proposed self-calibration method?

Key findings

• The paper provides a formal theoretical proof that two-view self-calibration of radial distortion is degenerate under forward motion due to an inherent ambiguity between radial distortion and scene depth.
• The proposed CNN-based method achieves a median root mean square error (RMSE) of 8.04 meters for keyframe trajectory estimation on the entire KITTI Raw test sequence, which is very close to the ground truth RMSE of 7.29 meters.
• While the hand-crafted method ('HandCrafted+OVP') failed in all five runs of ORB-SLAM on the KITTI Raw sequence, the proposed method enabled successful SLAM execution without significant drift.
• On YouTube test sequences, the proposed method successfully enabled ORB-SLAM to run on all three sequences, whereas the hand-crafted method failed on the last two due to initialization failure.
• Qualitative results show that the scene structure and camera trajectory reconstructed using the proposed method are significantly more accurate and less distorted than those from the hand-crafted baseline.
• The method generalizes well to real-world uncalibrated videos, demonstrating that deep supervision in the CNN leads to better performance than standard multi-task learning for intrinsic and distortion estimation.