[Paper Review] Attention-SLAM: A Visual Monocular SLAM Learning from Human Gaze

Jinquan Li, Ling Pei+6 more|arXiv (Cornell University)|Sep 15, 2020

Robotics and Sensor-Based Localization41 references5 citations

TL;DR

This paper proposes Attention-SLAM, a monocular visual SLAM system that enhances accuracy and robustness by integrating a human gaze-inspired saliency model (SalNavNet) to weight feature points during Bundle Adjustment. By mitigating center bias and leveraging saliency maps to prioritize salient objects, Attention-SLAM outperforms DSO, ORB-SLAM, and Salient DSO in trajectory accuracy and uncertainty reduction on the Salient Euroc dataset.

ABSTRACT

This paper proposes a novel simultaneous localization and mapping (SLAM) approach, namely Attention-SLAM, which simulates human navigation mode by combining a visual saliency model (SalNavNet) with traditional monocular visual SLAM. Most SLAM methods treat all the features extracted from the images as equal importance during the optimization process. However, the salient feature points in scenes have more significant influence during the human navigation process. Therefore, we first propose a visual saliency model called SalVavNet in which we introduce a correlation module and propose an adaptive Exponential Moving Average (EMA) module. These modules mitigate the center bias to enable the saliency maps generated by SalNavNet to pay more attention to the same salient object. Moreover, the saliency maps simulate the human behavior for the refinement of SLAM results. The feature points extracted from the salient regions have greater importance in optimization process. We add semantic saliency information to the Euroc dataset to generate an open-source saliency SLAM dataset. Comprehensive test results prove that Attention-SLAM outperforms benchmarks such as Direct Sparse Odometry (DSO), ORB-SLAM, and Salient DSO in terms of efficiency, accuracy, and robustness in most test cases.

Motivation & Objective

• To improve monocular SLAM accuracy and robustness by incorporating human-like attention mechanisms inspired by visual saliency.
• To address the limitation of traditional SLAM systems that treat all features equally, regardless of semantic relevance.
• To reduce uncertainty in pose estimation using information theory and saliency-based keyframe selection.
• To create a new benchmark dataset—Salient Euroc—labeled with saliency maps for navigation-focused SLAM evaluation.
• To investigate whether saliency models trained on non-navigation datasets hinder SLAM performance due to center bias.

Proposed method

• Proposes SalNavNet, a visual saliency model with a correlation module and adaptive Exponential Moving Average (EMA) to reduce center bias and improve temporal consistency in saliency prediction.
• Introduces a weighted Bundle Adjustment framework that assigns higher optimization weights to feature points in salient image regions, enhancing tracking stability.
• Uses information theory to select keyframes based on saliency-driven uncertainty reduction, improving efficiency and accuracy.
• Generates a new open-source dataset—Salient Euroc—by adding saliency annotations to the original Euroc dataset using SalNavNet.
• Employs SalGAN and SalEMA as baseline saliency models for comparison, analyzing their performance differences in SLAM context.
• Applies saliency maps as attention masks to guide feature selection and optimization, ensuring consistent tracking of salient objects across frames.

Experimental results

Research questions

• RQ1 Can a saliency model trained on general image datasets effectively improve monocular SLAM performance when applied to navigation tasks?
• RQ2 How does center bias in existing saliency models affect SLAM accuracy, particularly during object motion across the image frame?
• RQ3 Does incorporating saliency-based weighting in Bundle Adjustment reduce uncertainty in pose estimation compared to standard SLAM methods?
• RQ4 To what extent does the choice of saliency model training data impact the performance of a saliency-augmented SLAM system?
• RQ5 Can a saliency-aware SLAM system achieve higher accuracy and robustness than state-of-the-art benchmarks like ORB-SLAM and DSO?

Key findings

• Attention-SLAM achieves lower trajectory error than DSO, ORB-SLAM, and Salient DSO across most Euroc sequences, particularly in medium and easy difficulty datasets.
• The saliency maps generated by SalGAN—trained on SALICON—outperform those from SalEMA (trained on DHF1K) despite SalEMA’s use of LSTMs, due to SALICON’s weaker center bias.
• SalNavNet successfully mitigates center bias and enables consistent tracking of salient objects even when they move from center to edge of the image.
• The system reduces uncertainty in pose estimation by leveraging saliency-based keyframe selection, as validated through information-theoretic analysis.
• Motion blur and fast rotation degrade performance, especially on MH04, MH05, V103, and V203, due to poor feature extraction and saliency map misalignment.
• The open-source Salient Euroc dataset is released to support future research in saliency-driven visual SLAM.