[Paper Review] OmniSLAM: Omnidirectional Localization and Dense Mapping for Wide-baseline Multi-camera Systems

Changhee Won, Hochang Seok+3 more|arXiv (Cornell University)|Mar 18, 2020

Advanced Vision and Imaging46 references4 citations

TL;DR

This paper presents OmniSLAM, a robust visual SLAM and dense mapping system for wide-baseline omnidirectional multi-camera setups using fisheye lenses. It introduces lightweight, accurate deep networks for 360° depth estimation, integrates depth into visual odometry with improved feature matching and loop closing, and fuses results into a TSDF volume for high-fidelity 3D reconstruction, achieving state-of-the-art accuracy in both synthetic and real-world environments.

ABSTRACT

In this paper, we present an omnidirectional localization and dense mapping system for a wide-baseline multiview stereo setup with ultra-wide field-of-view (FOV) fisheye cameras, which has a 360 degrees coverage of stereo observations of the environment. For more practical and accurate reconstruction, we first introduce improved and light-weighted deep neural networks for the omnidirectional depth estimation, which are faster and more accurate than the existing networks. Second, we integrate our omnidirectional depth estimates into the visual odometry (VO) and add a loop closing module for global consistency. Using the estimated depth map, we reproject keypoints onto each other view, which leads to a better and more efficient feature matching process. Finally, we fuse the omnidirectional depth maps and the estimated rig poses into the truncated signed distance function (TSDF) volume to acquire a 3D map. We evaluate our method on synthetic datasets with ground-truth and real-world sequences of challenging environments, and the extensive experiments show that the proposed system generates excellent reconstruction results in both synthetic and real-world environments.

Motivation & Objective

• To address the limitations of traditional stereo systems in omnidirectional sensing, particularly limited field-of-view and blind spots.
• To develop a practical, efficient, and accurate dense mapping system for wide-baseline omnidirectional multi-camera rigs with ultra-wide FOV fisheye lenses.
• To improve depth estimation accuracy and speed for 360° panoramic scenes using lightweight deep neural networks.
• To integrate omnidirectional depth into visual odometry for enhanced pose estimation and global consistency via loop closing.
• To achieve high-quality 3D reconstruction by fusing depth maps and camera poses into a truncated signed distance function (TSDF) volume.

Proposed method

• Proposes improved and lightweight deep neural networks for omnidirectional depth estimation from fisheye images, reducing GPU memory and inference time while improving accuracy over prior methods.
• Integrates estimated depth maps into the ROVO visual odometry framework to enhance inter-view feature matching and triangulation by reprojecting keypoints across views.
• Implements a specialized loop closing module for the omnidirectional setup to correct trajectory drift and improve global consistency.
• Uses a truncated signed distance function (TSDF) volume to fuse omnidirectional depth maps and estimated rig poses into a coherent 3D reconstruction.
• Applies image preprocessing such as histogram equalization and gamma correction to improve depth estimation in low-contrast or textureless regions.
• Employs keyframe-based fusion with observation count filtering to reduce noise and improve map quality in challenging regions like large walls or floors.

Experimental results

Research questions

• RQ1 Can lightweight, accurate deep networks be developed for 360° depth estimation from wide-baseline fisheye cameras to improve efficiency and performance?
• RQ2 How does integrating omnidirectional depth into visual odometry improve pose estimation accuracy and robustness in complex environments?
• RQ3 To what extent does a dedicated loop closing module enhance global consistency in omnidirectional SLAM systems?
• RQ4 How well does the fusion of depth and pose into a TSDF volume produce high-fidelity 3D maps across diverse real-world and synthetic scenes?
• RQ5 Can the proposed system achieve state-of-the-art reconstruction quality while remaining computationally practical for real-time deployment?

Key findings

• The proposed OmniMVS+ and Tiny+ networks achieve higher accuracy and faster inference than OmniMVS [15], with reduced parameter count and lower GPU memory usage.
• ROVO+ with depth integration reduces trajectory drift significantly, outperforming ROVO [14] in both synthetic and real-world sequences.
• The addition of loop closing to ROVO+ further improves trajectory alignment, as shown by better correspondence with satellite maps in outdoor environments.
• On synthetic datasets, OmniMVS+ and Tiny+ achieve superior completeness and accuracy across all thresholds, especially at low thresholds (e.g., 0.1m).
• The system successfully reconstructs detailed 3D maps of challenging indoor and outdoor environments, including textureless regions and small objects, as demonstrated on IT/BT and Wangsimni datasets.
• Fusion with TSDF using observation count filtering and optimized voxel size (0.05m–0.15m) enhances map quality, particularly in large, low-texture areas.