[Paper Review] Spatio-Temporal Multisensor Calibration Based on Gaussian Processes Moving Object Tracking

Juraj Peršić, Luka Petrović+2 more|arXiv (Cornell University)|Apr 8, 2019

Gaussian Processes and Bayesian Inference29 references10 citations

TL;DR

This paper proposes a spatio-temporal multisensor calibration method using Gaussian process (GP)-estimated trajectories of moving objects to jointly estimate time delays and extrinsic parameters. By leveraging coordinate frame-invariant velocity magnitudes for time delay estimation and subsequent 3D point correspondence alignment, the method achieves sub-sampling-period accuracy (±3% of sampling interval) and extrinsic calibration with <0.1° angular and 3 mm translational error, validated in simulation and real-world experiments with cameras and motion capture systems.

ABSTRACT

Perception is one of the key abilities of autonomous mobile robotic systems, which often relies on fusion of heterogeneous sensors. Although this heterogeneity presents a challenge for sensor calibration, it is also the main prospect for reliability and robustness of autonomous systems. In this paper, we propose a method for multisensor calibration based on Gaussian processes (GPs) estimated moving object trajectories, resulting with temporal and extrinsic parameters. The appealing properties of the proposed temporal calibration method are: coordinate frame invariance, thus avoiding prior extrinsic calibration, theoretically grounded batch state estimation and interpolation using GPs, computational efficiency with O(n) complexity, leveraging data already available in autonomous robot platforms, and the end result enabling 3D point-to-point extrinsic multisensor calibration. The proposed method is validated both in simulations and real-world experiments. For real-world experiment we evaluated the method on two multisensor systems: an externally triggered stereo camera, thus having temporal ground truth readily available, and a heterogeneous combination of a camera and motion capture system. The results show that the estimated time delays are accurate up to a fraction of the fastest sensor sampling time.

Motivation & Objective

• To address the challenge of calibrating heterogeneous multisensor systems in autonomous robots without requiring prior extrinsic calibration.
• To enable accurate temporal calibration using only velocity magnitude trajectories, which are invariant to coordinate frame transformations.
• To achieve joint estimation of time delays and extrinsic parameters using GP-based trajectory interpolation and correspondence matching.
• To validate the method in both simulated and real-world environments with diverse sensor setups, including externally triggered cameras and camera–motion capture systems.
• To demonstrate scalability and computational efficiency with O(n) complexity for real-time or near-real-time application.

Proposed method

• The method uses Gaussian process regression to model continuous-time trajectories from asynchronous, noisy sensor measurements of a moving object.
• Velocity magnitude profiles from each sensor are extracted from the GP-estimated trajectories to enable coordinate frame-invariant time delay estimation.
• Time delay is estimated by aligning the velocity magnitude trajectories using a batch optimization framework with a smoothness prior.
• After time delay estimation, 3D point correspondences are generated by interpolating trajectories at aligned timestamps to estimate extrinsic parameters (rotation and translation).
• The method leverages existing sensor data from autonomous systems, avoiding additional calibration hardware or targets.
• Optimization is performed using the Ceres solver, with initialization based on GP-generated correspondences, achieving convergence in ~6 iterations.

Experimental results

Research questions

• RQ1 Can time delay estimation be achieved without prior extrinsic calibration by using only velocity magnitude trajectories?
• RQ2 How accurately can time delays be estimated using GP-based continuous-time trajectory modeling in the presence of asynchronous, noisy sensor data?
• RQ3 To what extent does the proposed method improve upon state-of-the-art approaches in temporal and extrinsic calibration accuracy?
• RQ4 What is the computational efficiency and scalability of the method for real-time applications in autonomous systems?
• RQ5 Can the method be applied to diverse sensor combinations, such as cameras and motion capture systems, without requiring specialized hardware or targets?

Key findings

• The method achieved time delay estimation with a mean error of -1.06×10⁻⁵ s and a standard deviation of 7.67×10⁻⁷ s in simulation, indicating sub-millisecond accuracy.
• In real-world experiments with externally triggered stereo cameras, the estimated time delays were accurate to within a fraction of the fastest sensor's sampling interval.
• For the camera–motion capture system setup, the method successfully estimated time delays and extrinsic parameters with a maximum angular deviation of 0.065° and 3 mm translation error from the mean.
• The simulation results showed a maximum deviation of 0.1° in Euler angles and 3 mm in translation for the estimated extrinsic parameters.
• The total calibration time averaged 94 ms per minute of trajectory data, with 49 ms for GP regression and 45 ms for optimization, demonstrating O(n) scalability.
• The method outperformed two state-of-the-art approaches in both temporal and extrinsic calibration accuracy, confirming its robustness and reliability.