[Paper Review] ROVO: Robust Omnidirectional Visual Odometry for Wide-baseline Wide-FOV Camera Systems

Hochang Seok, Jongwoo Lim|arXiv (Cornell University)|Feb 28, 2019

Robotics and Sensor-Based Localization22 references4 citations

TL;DR

This paper proposes ROVO, a robust omnidirectional visual odometry system for wide-baseline, wide-FOV fisheye camera rigs. It introduces a hybrid projection model to reduce distortion and improve feature matching, a multi-view P3P RANSAC algorithm for robust pose estimation, and online extrinsic calibration integrated into bundle adjustment, achieving high accuracy and stability in dynamic environments with real and synthetic datasets.

ABSTRACT

In this paper we propose a robust visual odometry system for a wide-baseline camera rig with wide field-of-view (FOV) fisheye lenses, which provides full omnidirectional stereo observations of the environment. For more robust and accurate ego-motion estimation we adds three components to the standard VO pipeline, 1) the hybrid projection model for improved feature matching, 2) multi-view P3P RANSAC algorithm for pose estimation, and 3) online update of rig extrinsic parameters. The hybrid projection model combines the perspective and cylindrical projection to maximize the overlap between views and minimize the image distortion that degrades feature matching performance. The multi-view P3P RANSAC algorithm extends the conventional P3P RANSAC to multi-view images so that all feature matches in all views are considered in the inlier counting for robust pose estimation. Finally the online extrinsic calibration is seamlessly integrated in the backend optimization framework so that the changes in camera poses due to shocks or vibrations can be corrected automatically. The proposed system is extensively evaluated with synthetic datasets with ground-truth and real sequences of highly dynamic environment, and its superior performance is demonstrated.

Motivation & Objective

• Address the challenges of feature matching and pose estimation in wide-baseline, wide-FOV fisheye camera systems due to extreme lens distortion and large viewpoint differences.
• Improve robustness in highly dynamic environments where moving objects dominate the field of view.
• Mitigate drift and errors caused by rig deformation or inaccurate initial extrinsic calibration during operation.
• Enable accurate metric ego-motion estimation using multiple overlapping fisheye views with minimal baseline loss.
• Develop a system that maintains high performance under real-world conditions such as rapid illumination changes, narrow passages, and high vehicle density.

Proposed method

• Proposes a hybrid projection model combining perspective and cylindrical projections to minimize distortion and maximize feature overlap across fisheye views.
• Uses the hybrid projection to enable continuous feature tracking and consistent descriptor computation across views.
• Introduces a multi-view P3P RANSAC algorithm that evaluates inliers across all views simultaneously, improving robustness in pose estimation.
• Integrates online extrinsic calibration into the local bundle adjustment framework to dynamically correct rig-to-camera pose errors due to shocks or vibrations.
• Employs a standard camera rig calibration with a large checkerboard for initial extrinsic parameter estimation.
• Applies the system to both synthetic datasets with ground-truth trajectories and real-world sequences collected from a vehicle-mounted rig.

Experimental results

Research questions

• RQ1 How can feature matching be improved in wide-baseline, wide-FOV fisheye camera systems with significant lens distortion?
• RQ2 Can a multi-view extension of P3P RANSAC enhance pose estimation robustness in highly dynamic scenes with moving objects?
• RQ3 To what extent does online extrinsic calibration reduce drift and improve accuracy in real-world deployments?
• RQ4 How does the hybrid projection model compare to standard fisheye or equirectangular projections in terms of feature matching quality?
• RQ5 Can the proposed system maintain high accuracy under extreme environmental conditions such as rapid lighting changes and narrow urban streets?

Key findings

• The hybrid projection model reduces average descriptor distance from 124.85 to 35.27 and increases inlier ratio from 21.53% to 73.16%, significantly improving feature matching reliability.
• With online extrinsic calibration, the system reduces trajectory error to near ground-truth levels even when starting with noisy extrinsic parameters (σ = 5°), converging to correct values within 100 frames.
• In the ParkingLot sequence, the system achieves accurate trajectory estimation without loop closing, demonstrating strong local consistency.
• In Wangsimni and Seongsu real-world sequences, the system successfully rejects outliers from moving vehicles and maintains accurate pose estimation in narrow, dynamic streets.
• Reprojection error and inlier ratio improve significantly with online calibration, approaching the performance of ground-truth extrinsic parameters.
• The system demonstrates robustness in challenging conditions such as heavy traffic, rapid illumination changes, and high dynamic range scenes, outperforming baseline methods in both synthetic and real datasets.