[Paper Review] Neural SLAM

Jingwei Zhang, Lei Tai+3 more|arXiv (Cornell University)|Jun 29, 2017

Reinforcement Learning in Robotics14 citations

TL;DR

This paper proposes Neural SLAM, a differentiable deep learning framework that embeds traditional SLAM procedures into the attention mechanism of an external memory network, enabling reinforcement learning agents to learn global environmental representations for effective long-term exploration. The method successfully enables agents to explore complex environments by integrating memory-based spatial reasoning with end-to-end training.

ABSTRACT

We present an approach for agents to learn representations of a global map from sensor data, to aid their exploration in new environments. To achieve this, we embed procedures mimicking that of traditional Simultaneous Localization and Mapping (SLAM) into the soft attention based addressing of external memory architectures, in which the external memory acts as an internal representation of the environment. This structure encourages the evolution of SLAM-like behaviors inside a completely differentiable deep neural network. We show that this approach can help reinforcement learning agents to successfully explore new environments where long-term memory is essential. We validate our approach in both challenging grid-world environments and preliminary Gazebo experiments. A video of our experiments can be found at: this https URL.

Motivation & Objective

• To enable reinforcement learning agents to learn global environmental representations in unseen environments.
• To address the challenge of long-term memory in exploration tasks where agents must retain spatial knowledge over extended sequences.
• To integrate traditional SLAM-like behaviors into a differentiable neural architecture using external memory.
• To validate the approach in both simulated grid-worlds and preliminary Gazebo robotics environments.

Proposed method

• The method uses a soft attention mechanism to address an external memory, which functions as an internal map of the environment.
• Traditional SLAM operations—such as localization and mapping—are embedded into the attention computation to guide memory updates.
• The entire architecture is fully differentiable, allowing end-to-end training via reinforcement learning.
• The memory is updated based on sensor observations, with attention gates determining which memory locations to read from or write to.
• The network is trained using a reinforcement learning objective that encourages exploration and map accuracy.
• The approach leverages the inductive bias of SLAM to shape the agent’s internal representation of space.

Experimental results

Research questions

• RQ1 Can a deep neural network learn to build and maintain a consistent global map through end-to-end training?
• RQ2 How well does the integration of SLAM-like operations into a memory-augmented network improve long-horizon exploration?
• RQ3 To what extent can the differentiable architecture support robust localization and mapping in novel environments?
• RQ4 Does the use of attention-based memory improve sample efficiency and generalization in exploration tasks?

Key findings

• The Neural SLAM framework enables agents to learn global maps in new environments through end-to-end training.
• The method successfully supports long-term memory and consistent spatial reasoning during exploration.
• The approach demonstrates effectiveness in challenging grid-world environments, showing improved exploration performance.
• Preliminary experiments in Gazebo validate the feasibility of the approach in simulated robotic settings.
• The integration of SLAM procedures into attention-based memory improves the agent's ability to navigate and map complex spaces.
• The differentiable architecture allows for stable training and meaningful internal representations of the environment.