[Paper Review] Learning to Navigate in Indoor Environments: from Memorizing to Reasoning

Liulong Ma, Yanjie Liu+2 more|arXiv (Cornell University)|Apr 15, 2019

Reinforcement Learning in Robotics22 references15 citations

TL;DR

This paper proposes a deep reinforcement learning-based navigation planner that enables mobile robots to reach unseen indoor goals using only RGB images and odometry, leveraging a stacked LSTM architecture to generalize beyond memorized paths. The model achieves zero-shot generalization to new targets in both simulation and real-world environments, demonstrating reasoning over visual observations rather than relying on memorization.

ABSTRACT

Autonomous navigation is an essential capability of smart mobility for mobile robots. Traditional methods must have the environment map to plan a collision-free path in workspace. Deep reinforcement learning (DRL) is a promising technique to realize the autonomous navigation task without a map, with which deep neural network can fit the mapping from observation to reasonable action through explorations. It should not only memorize the trained target, but more importantly, the planner can reason out the unseen goal. We proposed a new motion planner based on deep reinforcement learning that can arrive at new targets that have not been trained before in the indoor environment with RGB image and odometry only. The model has a structure of stacked Long Short-Term memory (LSTM). Finally, experiments were implemented in both simulated and real environments. The source code is available: https://github.com/marooncn/navbot.

Motivation & Objective

• To develop a map-free navigation system that generalizes to unseen goals in indoor environments.
• To enable robots to reason about visual observations rather than memorize specific paths.
• To design a motion planner that operates with only RGB images and odometry, avoiding reliance on pre-built maps.
• To validate the model's generalization capability in both simulated and real-world indoor settings.

Proposed method

• The planner uses a deep reinforcement learning framework to map observations to actions without requiring a pre-existing environment map.
• A stacked Long Short-Term Memory (LSTM) network processes sequential visual and odometric inputs to model temporal dependencies.
• The agent is trained via reinforcement learning to maximize sparse dense rewards for reaching goal locations.
• The architecture enables reasoning over visual context by maintaining a memory of past observations and actions.
• Training is conducted in a simulated environment before transfer to real-world testing with minimal domain randomization.
• The model is evaluated on its ability to reach novel, untrained goal locations not seen during training.

Experimental results

Research questions

• RQ1 Can a deep reinforcement learning agent navigate to new, untrained goals in indoor environments without prior memorization?
• RQ2 To what extent can an agent generalize from seen to unseen goal locations using only RGB and odometry?
• RQ3 How effective is a stacked LSTM architecture in enabling reasoning over visual sequences for navigation?
• RQ4 Does the proposed method achieve robust performance in real-world indoor settings without explicit map supervision?

Key findings

• The model successfully navigates to previously unseen goals in both simulation and real-world environments, demonstrating zero-shot generalization.
• The stacked LSTM architecture enables the agent to reason over sequential visual observations, improving generalization beyond memorization.
• The system achieves high success rates in reaching novel targets, even when the goal location is significantly different from training examples.
• The model generalizes effectively from simulation to real-world deployment with minimal domain adaptation.