[Paper Review] Towards Deployment of Deep-Reinforcement-Learning-Based Obstacle Avoidance into Conventional Autonomous Navigation Systems.

Linh Kästner, Teham Buiyan+4 more|arXiv (Cornell University)|Apr 8, 2021

Robotic Path Planning Algorithms20 references4 citations

TL;DR

This paper proposes integrating deep reinforcement learning (DRL)-based local planners into conventional navigation stacks to enable safe, efficient, and robust long-range navigation in mobile robots. By combining DRL’s adaptability with classical planning frameworks, the system overcomes DRL’s limitations in long-term memory and local minima, outperforming traditional planners in safety, efficiency, and robustness across evaluations.

ABSTRACT

Recently, mobile robots have become important tools in various industries, especially in logistics. Deep reinforcement learning emerged as an alternative planning method to replace overly conservative approaches and promises more efficient and flexible navigation. However, deep reinforcement learning approaches are not suitable for long-range navigation due to their proneness to local minima and lack of long term memory, which hinders its widespread integration into industrial applications of mobile robotics. In this paper, we propose a navigation system incorporating deep-reinforcement-learning-based local planners into conventional navigation stacks for long-range navigation. Therefore, a framework for training and testing the deep reinforcement learning algorithms along with classic approaches is presented. We evaluated our deep-reinforcement-learning-enhanced navigation system against various conventional planners and found that our system outperforms them in terms of safety, efficiency and robustness.

Motivation & Objective

• Address the limitations of deep reinforcement learning (DRL) in long-range navigation due to local minima and lack of long-term memory.
• Enable practical deployment of DRL-based obstacle avoidance in industrial mobile robotics by integrating it with conventional navigation systems.
• Develop a unified framework for training and testing DRL-based planners alongside classical planning algorithms.
• Improve navigation performance in terms of safety, efficiency, and robustness compared to conventional approaches.

Proposed method

• Design a hybrid navigation architecture that combines a conventional global planner with a DRL-based local planner.
• Train the DRL agent using reinforcement learning in simulated environments to learn obstacle avoidance policies.
• Integrate the trained DRL policy into a standard ROS-based navigation stack to ensure compatibility with existing systems.
• Use curriculum learning and reward shaping to stabilize training and improve sample efficiency.
• Implement a perception pipeline that provides consistent state observations (e.g., LiDAR scans) to the DRL agent.
• Evaluate the DRL-enhanced system using standardized benchmarks and compare it directly with classical planners like DWA and TEB.

Experimental results

Research questions

• RQ1 Can a DRL-based local planner be effectively integrated into a conventional navigation stack to enable long-range navigation?
• RQ2 How does the DRL-enhanced system compare to classical planners in terms of safety, efficiency, and robustness?
• RQ3 To what extent does the DRL component reduce reliance on conservative, heuristic-based behaviors in obstacle avoidance?
• RQ4 Does the integration preserve system stability and compatibility with existing industrial robotics software stacks?

Key findings

• The DRL-enhanced navigation system outperformed conventional planners in terms of navigation success rate and path efficiency.
• The system demonstrated improved robustness in dynamic and cluttered environments where classical planners failed or exhibited overly conservative behavior.
• The integration of DRL into the conventional stack enabled effective long-range navigation without the typical DRL pitfalls of local minima and memory limitations.
• The framework successfully maintained compatibility with standard ROS navigation stacks, enabling practical deployment in industrial settings.