[Paper Review] A Lifelong Learning Approach to Mobile Robot Navigation

Bo Liu, Xuesu Xiao+1 more|arXiv (Cornell University)|Jul 28, 2020

Robotic Path Planning Algorithms35 references8 citations

TL;DR

This paper proposes Lifelong Learning for Navigation (LLfN), a self-supervised framework enabling a mobile robot to continuously improve its navigation performance through experience in a single environment while retaining skills in previously encountered environments. The method uses a complementary policy trained on suboptimal behaviors identified via a heuristic scoring function, achieving continual learning without catastrophic forgetting, and is fully implemented onboard a physical robot with limited computational resources, outperforming sequential training and matching individual models in efficiency.

ABSTRACT

This paper presents a self-improving lifelong learning framework for a mobile robot navigating in different environments. Classical static navigation methods require environment-specific in-situ system adjustment, e.g. from human experts, or may repeat their mistakes regardless of how many times they have navigated in the same environment. Having the potential to improve with experience, learning-based navigation is highly dependent on access to training resources, e.g. sufficient memory and fast computation, and is prone to forgetting previously learned capability, especially when facing different environments. In this work, we propose Lifelong Learning for Navigation (LLfN) which (1) improves a mobile robot's navigation behavior purely based on its own experience, and (2) retains the robot's capability to navigate in previous environments after learning in new ones. LLfN is implemented and tested entirely onboard a physical robot with a limited memory and computation budget.

Motivation & Objective

• To address the limitations of classical static navigation systems that cannot improve with experience or adapt to new environments.
• To overcome catastrophic forgetting in learning-based navigation when transitioning between different environments.
• To develop a self-supervised, lifelong learning framework that enhances navigation performance through robot-generated experience, without requiring human demonstrations or offboard computation.
• To implement and validate the framework entirely onboard a physical robot under real-time constraints of limited memory and processing power.

Proposed method

• The framework initializes with a static sampling-based model predictive control policy (DWA) that does not improve with experience.
• Suboptimal actions are identified using a heuristic scoring function based on motion duration and proximity to obstacles.
• A complementary policy is trained online using a buffer of past experiences, prioritizing similar scenarios where good actions were performed.
• The learning process is self-supervised: no human demonstrations or reward shaping are used; instead, the robot learns from its own mistakes.
• The method ensures backward transfer by retaining performance on previous environments through experience replay and selective fine-tuning.
• The entire system is deployed onboard a physical robot using only its onboard CPU and memory, with training taking under two minutes per environment.

Experimental results

Research questions

• RQ1 Can a mobile robot self-improve its navigation performance over time without human intervention or external rewards?
• RQ2 Can a learning-based navigation system avoid catastrophic forgetting when exposed to multiple distinct environments in sequence?
• RQ3 To what extent can a robot leverage its own experience to refine navigation policies in real-time with limited computational resources?
• RQ4 How does the performance of the lifelong learning framework compare to sequential training and individual models trained per environment?

Key findings

• LLfN achieved an average navigation time of 28.32 ± 1.85 seconds in Environment 1, outperforming the Sequentially Trained model (39.11 ± 6.45 s) and matching the Individual Models (30.17 ± 0.97 s).
• In Environment 2, LLfN reduced average time to 19.98 ± 1.07 seconds, significantly outperforming the Sequentially Trained model (49.41 ± 13.94 s) and matching Individual Models (23.41 ± 0.66 s).
• In Environment 3, LLfN achieved 21.80 ± 1.51 seconds, comparable to Individual Models (21.12 ± 1.17 s) and substantially better than the Sequentially Trained model (21.12 ± 1.17 s).
• LLfN demonstrated strong backward transfer, performing better than individual models in Environment 1 and 2, suggesting improved generalization from diverse experience.
• The method achieved zero collisions and minimal recovery behaviors (0/0) in all environments, with the lowest standard deviation among all methods in Environment 1.
• Despite using only onboard computation, LLfN completed training in under two minutes per environment, confirming real-time feasibility on resource-constrained hardware.