[Paper Review] DF-SLAM: A Deep-Learning Enhanced Visual SLAM System based on Deep Local Features

Rong Kang, Jieqi Shi+3 more|arXiv (Cornell University)|Jan 22, 2019

Robotics and Sensor-Based LocalizationEngineering53 references33 citations

TL;DR

DF-SLAM substitutes hand-crafted local features with learned deep descriptors via a shallow network, enabling real-time SLAM with improved robustness and efficiency across challenging scenes.

ABSTRACT

As the foundation of driverless vehicle and intelligent robots, Simultaneous Localization and Mapping(SLAM) has attracted much attention these days. However, non-geometric modules of traditional SLAM algorithms are limited by data association tasks and have become a bottleneck preventing the development of SLAM. To deal with such problems, many researchers seek to Deep Learning for help. But most of these studies are limited to virtual datasets or specific environments, and even sacrifice efficiency for accuracy. Thus, they are not practical enough. We propose DF-SLAM system that uses deep local feature descriptors obtained by the neural network as a substitute for traditional hand-made features. Experimental results demonstrate its improvements in efficiency and stability. DF-SLAM outperforms popular traditional SLAM systems in various scenes, including challenging scenes with intense illumination changes. Its versatility and mobility fit well into the need for exploring new environments. Since we adopt a shallow network to extract local descriptors and remain others the same as original SLAM systems, our DF-SLAM can still run in real-time on GPU.

Motivation & Objective

• Motivate replacing non-geometric SLAM modules with learned local features to improve data association and robustness.
• Demonstrate that a shallow neural descriptor can replace traditional features without sacrificing real-time performance.
• Show that DF-SLAM achieves better efficiency and stability across diverse datasets compared to traditional SLAM systems.

Proposed method

• Use a shallow TFeat-based triplet network to produce 128-D L2-normalized local descriptors for keypoint patches.
• Replace traditional descriptors (e.g., ORB, SIFT) with learned deep descriptors in the standard SLAM pipeline.
• Employ a Bag-of-Words visual vocabulary (DBoW) trained on DF-Descriptors to enable fast frame retrieval and relocalization/loop closure.
• Maintain the original SLAM pipeline structure (tracking, local mapping, loop closing) to preserve comparability and efficiency.
• Ensure real-time performance by keeping the descriptor extractor lightweight and leveraging GPU acceleration.

Experimental results

Research questions

• RQ1 Can learned local feature descriptors substitute hand-crafted descriptors in a classic SLAM pipeline without losing real-time performance?
• RQ2 Do shallow network-based descriptors improve data association, robustness to illumination/texture changes, and loop closure stability in SLAM?
• RQ3 What is the impact of using DF-SLAM descriptors on localization accuracy and system stability across diverse datasets?
• RQ4 How does the DF-SLAM approach compare to traditional ORB-SLAM2 in challenging conditions (illumination changes, low texture) and with/without loop closure?

Key findings

• DF-SLAM runs at 10–15 fps on GPU with a single-frame descriptor extraction time of 0.09 s (1200 keypoints).
• Learned descriptors outperform traditional ones in matching, retrieval, and verification tasks on HPatches benchmarks.
• Across EuRoC and TUM datasets, DF-SLAM shows improved robustness and stability, particularly in sequences lacking loops or with challenging lighting and motion conditions.
• DF-SLAM achieves better or comparable accuracy to ORB-SLAM2 and demonstrates reduced drift in long, loop-sparse trajectories while maintaining real-time operation.
• A shallow descriptor network (two conv layers plus a 128-D output) with hard negative mining (HardTFeat_HD) improves matching performance versus the base TFeat.