[论文解读] Aeolus Ocean -- A simulation environment for the autonomous COLREG-compliant navigation of Unmanned Surface Vehicles using Deep Reinforcement Learning and Maritime Object Detection

Andrew Alexander Vekinis, Stavros Perantonis|arXiv (Cornell University)|Jul 13, 2023

Maritime Navigation and SafetyEngineering被引用 3

一句话总结

本文提出Aeolus Ocean，一个用于训练符合COLREGs规则的无人水面航行器（USV）基于深度强化学习（DRL）自主导航的高保真仿真环境。该系统集成海上目标检测以实现情境感知，并在真实气象与光照条件下成功实现了开阔海域及复杂会遇场景下的航路点导航，其仿真器和训练好的智能体的二进制可执行文件已公开发布。

ABSTRACT

Heading towards navigational autonomy in unmanned surface vehicles (USVs) in the maritime sector can fundamentally lead towards safer waters as well as reduced operating costs, while also providing a range of exciting new capabilities for oceanic research, exploration and monitoring. However, achieving such a goal is challenging. USV control systems must, safely and reliably, be able to adhere to the international regulations for preventing collisions at sea (COLREGs) in encounters with other vessels as they navigate to a given waypoint while being affected by realistic weather conditions, either during the day or at night. To deal with the multitude of possible scenarios, it is critical to have a virtual environment that is able to replicate the realistic operating conditions USVs will encounter, before they can be implemented in the real world. Such "digital twins" form the foundations upon which Deep Reinforcement Learning (DRL) and Computer Vision (CV) algorithms can be used to develop and guide USV control systems. In this paper we describe the novel development of a COLREG-compliant DRL-based collision avoidant navigational system with CV-based awareness in a realistic ocean simulation environment. The performance of the trained autonomous Agents resulting from this approach is evaluated in several successful navigations to set waypoints in both open sea and coastal encounters with other vessels. A binary executable version of the simulator with trained agents is available at https://github.com/aavek/Aeolus-Ocean

研究动机与目标

• 开发一个真实、高保真的仿真环境，用于在复杂海洋场景中训练自主USV。
• 通过深度强化学习（DRL）实现符合COLREGs规则的导航，并借助海上目标检测实现实时情境感知。
• 验证训练好的DRL智能体在多样化导航任务中的性能，包括开阔海域及与其他船只的沿海会遇场景。
• 提供仿真器和训练智能体的公开二进制版本，以支持可复现性与进一步研究。
• 通过建模真实的天气、光照及动态船舶交互，弥合仿真与实际部署之间的差距。

提出的方法

• 仿真环境Aeolus Ocean复现了包括风力、海浪以及昼夜周期在内的真实海洋环境。
• 通过奖励函数优先考虑COLREGs合规性，训练DRL智能体导航USV到达目标航路点，同时避免碰撞。
• 通过计算机视觉流程集成海上目标检测，以检测和跟踪其他船只，实现实时情境感知。
• DRL智能体处理来自模拟摄像头的视觉观测以做出导航决策，确保符合COLREGs规则。
• 该环境支持开阔海域与沿海导航场景，包含遵循真实交通模式的动态非玩家控制船只。
• 发布包含训练智能体的仿真器二进制可执行文件，支持即刻部署与基准测试。

实验结果

研究问题

• RQ1 基于DRL的导航系统是否能在复杂、动态的海洋环境中实现符合COLREGs规则的行为？
• RQ2 海上目标检测的集成在多大程度上提升了自主USV实现实时情境感知的能力？
• RQ3 DRL智能体是否能在多样化场景中实现泛化，包括开阔海域导航及与其他船只的近距离会遇？
• RQ4 仿真环境的真实性在多大程度上影响了训练智能体的泛化能力与鲁棒性？
• RQ5 训练好的智能体在真实气象与光照条件下，能否实现安全且高效的航路点导航？

主要发现

• DRL智能体在开阔海域与沿海场景中均成功导航至目标航路点，且未违反COLREGs规则。
• 海上目标检测的集成显著提升了智能体实时探测与响应附近船只的能力。
• 智能体在多种天气条件下表现出稳健性能，包括白天与夜间操作。
• 仿真环境支持稳定且可复现的训练，训练好的智能体在未见场景中也表现出良好泛化能力。
• 发布包含训练智能体的二进制可执行文件，便于在真实测试中立即使用，并推动进一步研究。