[Paper Review] CNN-SVO: Improving the Mapping in Semi-Direct Visual Odometry Using Single-Image Depth Prediction

Shing Yan Loo, Ali Amiri+3 more|arXiv (Cornell University)|Oct 1, 2018

Robotics and Sensor-Based Localization14 references8 citations

TL;DR

This paper proposes CNN-SVO, an enhancement to Semi-Direct Visual Odometry (SVO) that improves map point initialization by leveraging single-image depth prediction from a deep neural network. By initializing map points with low depth uncertainty—using predicted depth as the mean and a small variance—the method reduces erroneous feature matching and accelerates convergence to true depth, resulting in more robust and accurate camera tracking, especially in challenging lighting conditions.

ABSTRACT

Reliable feature correspondence between frames is a critical step in visual odometry (VO) and visual simultaneous localization and mapping (V-SLAM) algorithms. In comparison with existing VO and V-SLAM algorithms, semi-direct visual odometry (SVO) has two main advantages that lead to state-of-the-art frame rate camera motion estimation: direct pixel correspondence and efficient implementation of probabilistic mapping method. This paper improves the SVO mapping by initializing the mean and the variance of the depth at a feature location according to the depth prediction from a single-image depth prediction network. By significantly reducing the depth uncertainty of the initialized map point (i.e., small variance centred about the depth prediction), the benefits are twofold: reliable feature correspondence between views and fast convergence to the true depth in order to create new map points. We evaluate our method with two outdoor datasets: KITTI dataset and Oxford Robotcar dataset. The experimental results indicate that the improved SVO mapping results in increased robustness and camera tracking accuracy.

Motivation & Objective

• Address the limitation in SVO where map points are initialized with high depth uncertainty, leading to unreliable feature correspondence.
• Improve the robustness and accuracy of camera motion estimation in semi-direct visual odometry under varying lighting conditions.
• Leverage deep learning-based single-image depth prediction to provide accurate depth priors for map point initialization.
• Reduce the search range for feature matching by constraining depth uncertainty, thereby enhancing matching reliability.
• Enable real-time performance while maintaining high accuracy on challenging outdoor datasets like KITTI and Oxford Robotcar.

Proposed method

• Integrate a pre-trained single-image depth prediction network (e.g., trained on stereo datasets) to estimate depth maps for keyframes.
• Use the predicted depth as the mean for initializing map points, with a small variance to represent low depth uncertainty.
• Apply probabilistic depth-filters in the mapping thread to update depth estimates based on observations from multiple views.
• Limit feature matching search along the epipolar line to a depth interval centered on the predicted depth, reducing false matches.
• Use local bundle adjustment (BA) to refine camera poses and map points when new keyframes are added.
• Apply scale correction to depth predictions using ground truth poses when available, particularly for the Oxford Robotcar dataset.

Experimental results

Research questions

• RQ1 Can single-image depth prediction reduce depth uncertainty during map point initialization in SVO, leading to more reliable feature correspondence?
• RQ2 How does the use of depth priors from a deep neural network affect the convergence speed and accuracy of map point depth estimation?
• RQ3 To what extent does the proposed method improve camera tracking robustness in challenging lighting conditions such as overexposure or underexposure?
• RQ4 What is the impact of depth prediction uncertainty on the overall performance of visual odometry in real-world outdoor datasets?
• RQ5 Can the integration of depth priors enable real-time operation while maintaining high accuracy on high-frame-rate sequences?

Key findings

• CNN-SVO achieves improved robustness and camera tracking accuracy on both the KITTI and Oxford Robotcar datasets compared to standard SVO.
• The method reduces depth uncertainty during map point initialization, enabling faster convergence to true depth values.
• On the KITTI dataset, the scale of the odometry output is close to absolute scale, with values ranging from 0.921 to 1.1876 across sequences.
• On the Oxford Robotcar dataset, the scale of the odometry is between 0.8953 and 0.9737 after applying scale correction, indicating reasonable alignment with ground truth.
• Despite higher computational cost for local BA and depth prediction (around 29ms and 37ms respectively), CNN-SVO runs at 16 FPS on Oxford Robotcar and 10 FPS on KITTI, demonstrating real-time feasibility.
• The illumination invariance of the depth prediction network allows CNN-SVO to maintain performance in HDR conditions, overcoming a key limitation of original SVO.