[Paper Review] CNN Feature boosted SeqSLAM for Real-Time Loop Closure Detection

Dongdong Bai, Chaoqun Wang+3 more|arXiv (Cornell University)|Apr 17, 2017

Robotics and Sensor-Based Localization13 references3 citations

TL;DR

This paper proposes SeqCNNSLAM, a real-time loop closure detection method that fuses pre-trained CNN features (particularly pool5 layer) with sequence-based matching to achieve robustness against both viewpoint and condition changes. It introduces A-SeqCNNSLAM for runtime acceleration by limiting matching ranges using temporal priors, and O-SeqCNNSLAM for online parameter adjustment, achieving 4–6× speedup with minimal accuracy loss while maintaining state-of-the-art performance on benchmark datasets.

ABSTRACT

Loop closure detection (LCD) is an indispensable part of simultaneous localization and mapping systems (SLAM); it enables robots to produce a consistent map by recognizing previously visited places. When robots operate over extended periods, robustness to viewpoint and condition changes as well as satisfactory real-time performance become essential requirements for a practical LCD system. This paper presents an approach to directly utilize the outputs at the intermediate layer of a pre-trained convolutional neural network (CNN) as image descriptors. The matching location is determined by matching the image sequences through a method called SeqCNNSLAM. The utility of SeqCNNSLAM is comprehensively evaluated in terms of viewpoint and condition invariance. Experiments show that SeqCNNSLAM outperforms state-of-the-art LCD systems, such as SeqSLAM and Change Removal, in most cases. To allow for the real-time performance of SeqCNNSLAM, an acceleration method, A-SeqCNNSLAM, is established. This method exploits the location relationship between the matching images of adjacent images to reduce the matching range of the current image. Results demonstrate that acceleration of 4-6 is achieved with minimal accuracy degradation, and the method's runtime satisfies the real-time demand. To extend the applicability of A-SeqCNNSLAM to new environments, a method called O-SeqCNNSLAM is established for the online adjustment of the parameters of A-SeqCNNSLAM.

Motivation & Objective

• To address the challenge of robust loop closure detection under combined viewpoint and condition changes in long-term robotic navigation.
• To improve real-time performance of CNN-based loop closure detection by reducing computational complexity without sacrificing accuracy.
• To enable adaptation of the algorithm to new or unseen environments through online parameter adjustment.
• To evaluate the effectiveness of intermediate CNN features (especially pool5) in enhancing invariance to environmental and viewpoint variations.

Proposed method

• Utilizes the pool5 layer of a pre-trained CNN (e.g., AlexNet) as image descriptors to capture invariant features under viewpoint and condition changes.
• Applies sequence-based matching (SeqSLAM) to compare image sequences using CNN descriptors, enhancing robustness to environmental variations.
• Introduces A-SeqCNNSLAM, which reduces the matching range for each image by leveraging the sequential relationship between adjacent images, thus accelerating computation.
• Employs a dynamic parameter adjustment mechanism in O-SeqCNNSLAM that updates the number of matching ranges (K) online based on historical matching locations and scene change detection.
• Uses a change degree metric based on Euclidean distances between CNN descriptors to detect significant scene shifts and trigger K reinitialization.
• Employs a sliding window of the last 10 images to compute the image matching label (IML) and dynamically set K to the maximum IML value when scene stability is detected.

Experimental results

Research questions

• RQ1 Can intermediate CNN features (e.g., pool5) improve loop closure detection robustness under both viewpoint and condition changes?
• RQ2 How effective is sequence-based matching with CNN descriptors in outperforming traditional methods like SeqSLAM and Change Removal?
• RQ3 To what extent can matching range reduction via temporal priors accelerate loop closure detection with minimal accuracy degradation?
• RQ4 Can online parameter adjustment of the matching range (K) enhance adaptability to new environments without retraining?

Key findings

• SeqCNNSLAM outperforms state-of-the-art methods such as SeqSLAM and Change Removal in terms of robustness to both viewpoint and condition changes across multiple datasets.
• A-SeqCNNSLAM achieves 4–6× acceleration over the baseline SeqCNNSLAM while maintaining comparable accuracy, with runtime satisfying real-time requirements.
• O-SeqCNNSLAM achieves performance comparable to A-SeqCNNSLAM on the Nordland dataset with 12.5% shift, demonstrating effective online adaptation to new environments.
• The online adjustment of K in O-SeqCNNSLAM successfully maintains performance during scene transitions, with reinitialization triggered when the change degree metric exceeds 1.1 or falls below 0.9.
• Runtime for O-SeqCNNSLAM with ds=80 was 121.494s and with ds=100 was 151.932s, showing scalability and real-time feasibility.