[论文解读] Visual Exploration and Energy-aware Path Planning via Reinforcement Learning

Amir Niaraki, Jeremy Roghair+1 more|arXiv (Cornell University)|Sep 26, 2019

Robotic Path Planning Algorithms参考文献 30被引用 5

一句话总结

本文提出了一种基于强化学习的能量感知视觉探索与路径规划方法，通过将风致阻力建模为自适应负奖励，动态平衡目标检测准确率与电池寿命。在模拟实验中，该方法在强风条件下检测到的目标数量比完整覆盖规划方法高出最多4倍，在二维和三维飞行场景中均表现出更高的路径效率与覆盖度，优于基线方法。

ABSTRACT

Visual exploration and smart data collection via autonomous vehicles is an attractive topic in various disciplines. Disturbances like wind significantly influence both the power consumption of the flying robots and the performance of the camera. We propose a reinforcement learning approach which combines the effects of the power consumption and the object detection modules to develop a policy for object detection in large areas with limited battery life. The learning model enables dynamic learning of the negative rewards of each action based on the drag forces that is resulted by the motion of the flying robot with respect to the wind field. The algorithm is implemented in a near-real world simulation environment both for the planar motion and flight in different altitudes. The trained agent often performed a trade-off between detecting the objects with high accuracy and increasing the area coverage within its battery life. The developed exploration policy outperformed the complete coverage algorithm by minimizing the traveled path while finding the target objects. The performance of the algorithms under various wind fields was evaluated in planar and 3D motion. During an exploration task with sparsely distributed goals and within a UAV's battery life, the proposed architecture could detect more than twice the amount of goal objects compared to the coverage path planning algorithm in moderate wind field. In high wind intensities, the energy-aware algorithm could detect 4 times the amount of goal objects when compared to its complete coverage counterpart.

研究动机与目标

• 解决自主无人机在大范围视觉探索过程中电池寿命有限的挑战。
• 将功耗与目标检测性能整合到统一的强化学习策略中。
• 将风致阻力建模为动态奖励，以提升飞行过程中的能量效率。
• 优化路径规划，以在最小化行驶距离与能耗的同时最大化目标检测数量。
• 在二维与三维飞行环境中，评估不同风速条件下的性能表现。

提出的方法

• 采用深度Q网络（DQN）强化学习框架，训练智能体根据视觉与环境状态观测选择动作。
• 将风致阻力建模为动态负奖励，根据相对于风场的运动状态调整学习信号。
• 状态空间包括位置、速度、风向量以及基于CNN检测器的目标检测置信度。
• 动作空间由二维与三维运动中的离散控制输入（速度与方向）组成。
• 奖励函数结合了目标检测准确率、路径效率与能耗成本，并引入风速相关的惩罚项。
• 训练在具有不同风速强度与目标分布的近真实感仿真环境中进行。

实验结果

研究问题

• RQ1 如何利用强化学习在无人机探索中平衡目标检测准确率与能效？
• RQ2 将风致阻力建模为动态奖励在多大程度上提升了路径规划性能？
• RQ3 与基于完整覆盖的路径规划相比，所提方法在目标检测数量与能耗方面表现如何？
• RQ4 风速强度对能量感知探索策略性能有何影响？
• RQ5 该智能体能否在二维与三维飞行中，对不同的风场与目标分布实现良好泛化？

主要发现

• 在中等风速条件下，所提方法检测到的目标数量超过完整覆盖路径规划算法的两倍以上。
• 在高风速条件下，能量感知算法检测到的目标数量是基于覆盖的基线方法的四倍。
• 该智能体在区域覆盖与能效之间实现了显著的权衡，减少了路径长度，同时保持了高检测准确率。
• 模型在不同风速条件下表现出强鲁棒性，通过自适应奖励设计最小化了性能下降。
• 仿真结果证实，将风动力学整合到奖励函数中可提升长期能效与检测产量。
• 该方法在二维与三维运动场景中均优于基线覆盖策略，尤其在目标稀疏分布场景下表现更优。