[论文解读] AI based Algorithms of Path Planning, Navigation and Control for Mobile Ground Robots and UAVs

Jian Zhang|arXiv (Cornell University)|Oct 3, 2021

Robotic Path Planning Algorithms参考文献 265被引用 5

一句话总结

本文提出了一种用于移动地面机器人和无人机的混合人工智能路径规划与导航框架，通过将反应式导航与Q-learning相结合，实现在静态和动态环境中无碰撞、最优的路径规划。该方法在模拟的2D和3D环境中通过强化学习实现了高效、实时的路径规划，并具备强大的避障能力，性能显著优于基线方法。

ABSTRACT

As the demands of autonomous mobile robots are increasing in recent years, the requirement of the path planning/navigation algorithm should not be content with the ability to reach the target without any collisions, but also should try to achieve possible optimal or suboptimal path from the initial position to the target according to the robot's constrains in practice. This report investigates path planning and control strategies for mobile robots with machine learning techniques, including ground mobile robots and flying UAVs. In this report, the hybrid reactive collision-free navigation problem under an unknown static environment is investigated firstly. By combining both the reactive navigation and Q-learning method, we intend to keep the good characteristics of reactive navigation algorithm and Q-learning and overcome the shortcomings of only relying on one of them. The proposed method is then extended into 3D environments. The performance of the mentioned strategies are verified by extensive computer simulations, and good results are obtained. Furthermore, the more challenging dynamic environment situation is taken into our consideration. We tackled this problem by developing a new path planning method that utilizes the integrated environment representation and reinforcement learning. Our novel approach enables to find the optimal path to the target efficiently and avoid collisions in a cluttered environment with steady and moving obstacles. The performance of these methods is compared with other different aspects.

研究动机与目标

• 解决纯反应式导航和Q-learning在自主移动机器人路径规划中的局限性。
• 开发一种混合算法，结合反应式导航的实时响应能力与Q-learning的长期优化能力。
• 将该方法扩展至3D环境，适用于无人机及复杂障碍物场景。
• 实现在同时存在静止和移动障碍物的动态环境中的高效路径规划。
• 通过在多样化环境条件下进行广泛仿真，验证该方法的有效性。

提出的方法

• 混合架构将反应式导航用于即时避障，同时利用Q-learning实现长期路径优化。
• Q-learning组件在模拟环境中进行训练，以学习导航的最优状态-动作值。
• 将环境表示与强化学习相结合，以建模静态和动态障碍物。
• 通过状态空间离散化表示机器人的位置、目标位置以及障碍物位置，作为Q-learning的输入。
• 通过将状态空间和动作空间扩展以包含高度和方向，将算法推广至3D环境。
• 通过在不同障碍物密度和动态特性下进行广泛计算机仿真，评估性能。

实验结果

研究问题

• RQ1 如何有效结合反应式导航与Q-learning，以提升在未知静态环境中的路径规划性能？
• RQ2 与纯反应式或仅Q-learning的方法相比，该混合方法在路径最优性和碰撞避免方面的性能增益如何？
• RQ3 该混合方法能否成功扩展至无人机的3D环境，同时保持实时性能？
• RQ4 与现有基于强化学习的规划器相比，该方法在复杂环境中的动态障碍物处理能力如何？
• RQ5 影响仿真中收敛性和鲁棒性的关键设计因素有哪些？

主要发现

• 与仅使用反应式方法相比，该混合方法在未知静态环境中实现了无碰撞导航，并显著提升了路径最优性。
• 在3D仿真中，该方法成功为无人机规划出最优路径，同时避开了静态和动态障碍物。
• 将环境表示与强化学习相结合，实现了对障碍物配置变化的高效适应。
• 在多种仿真场景中，该方法在路径长度和成功率方面均优于基线的Q-learning和反应式方法。
• 该方法在包含移动障碍物的复杂、杂乱环境中表现出稳定的收敛性和实时性能。