[论文解读] Research on reinforcement learning based warehouse robot navigation algorithm in complex warehouse layout

Keqin Li, Lipeng Liu+6 more|arXiv (Cornell University)|Nov 9, 2024

Advanced Manufacturing and Logistics Optimization被引用 6

一句话总结

本文提出一种 Proximal Policy Optimization–Dijkstra (PP-D) 框架，将基于 PPO 的局部策略学习与 Dijkstra 的全局路径规划相结合，以在复杂布局中改善仓库机器人导航。

ABSTRACT

In this paper, how to efficiently find the optimal path in complex warehouse layout and make real-time decision is a key problem. This paper proposes a new method of Proximal Policy Optimization (PPO) and Dijkstra's algorithm, Proximal policy-Dijkstra (PP-D). PP-D method realizes efficient strategy learning and real-time decision making through PPO, and uses Dijkstra algorithm to plan the global optimal path, thus ensuring high navigation accuracy and significantly improving the efficiency of path planning. Specifically, PPO enables robots to quickly adapt and optimize action strategies in dynamic environments through its stable policy updating mechanism. Dijkstra's algorithm ensures global optimal path planning in static environment. Finally, through the comparison experiment and analysis of the proposed framework with the traditional algorithm, the results show that the PP-D method has significant advantages in improving the accuracy of navigation prediction and enhancing the robustness of the system. Especially in complex warehouse layout, PP-D method can find the optimal path more accurately and reduce collision and stagnation. This proves the reliability and effectiveness of the robot in the study of complex warehouse layout navigation algorithm.

研究动机与目标

• 解决在复杂仓库布局中实现高效且准确的路径查找。
• 在动态环境中实现实时决策。
• 提升导航鲁棒性，减少碰撞和停滞。

提出的方法

• 应用 Proximal Policy Optimization (PPO)，实现稳定、快速的策略更新并适应动态环境。
• 在静态环境中使用 Dijkstra 算法进行全局最优路径规划。
• 将 PPO 与 Dijkstra 集成到 Proximal policy-Dijkstra (PP-D) 框架中，以在局部学习和全局规划之间取得平衡。
• 评估 PP-D 相对于传统算法在导航精度和鲁棒性方面的改进。

实验结果

研究问题

• RQ1 在复杂仓库布局中，PP-D 在导航精度方面的表现如何？
• RQ2 与传统方法相比，PP-D 是否提升鲁棒性并减少碰撞和停滞？
• RQ3 在这种设置下，实时决策（PPO）与全局最优性（Dijkstra）之间有哪些权衡？

主要发现

• 相较于传统算法，PP-D 提高了导航精度和鲁棒性。
• 在复杂布局中，PP-D 能更准确地找到最优路径。
• PP-D 减少了碰撞和停滞事件，提升了可靠性。
• PPO 实现快速适应以进行实时决策，而 Dijkstra 提供路径规划的全局最优性。