[论文解读] A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Network

Jiqian Dong, Sikai Chen+3 more|arXiv (Cornell University)|Oct 12, 2020

Autonomous Vehicle Technology and Safety参考文献 25被引用 39

一句话总结

本文提出一种基于深度强化学习的框架，融合协作感知与图卷积Q网络（GCQ），为多辆自动驾驶车辆（CAVs）提供安全、协作的变道决策，适用于集中式 RSU/云部署。

ABSTRACT

Connected Autonomous Vehicle (CAV) Network can be defined as a collection of CAVs operating at different locations on a multilane corridor, which provides a platform to facilitate the dissemination of operational information as well as control instructions. Cooperation is crucial in CAV operating systems since it can greatly enhance operation in terms of safety and mobility, and high-level cooperation between CAVs can be expected by jointly plan and control within CAV network. However, due to the highly dynamic and combinatory nature such as dynamic number of agents (CAVs) and exponentially growing joint action space in a multiagent driving task, achieving cooperative control is NP hard and cannot be governed by any simple rule-based methods. In addition, existing literature contains abundant information on autonomous driving's sensing technology and control logic but relatively little guidance on how to fuse the information acquired from collaborative sensing and build decision processor on top of fused information. In this paper, a novel Deep Reinforcement Learning (DRL) based approach combining Graphic Convolution Neural Network (GCN) and Deep Q Network (DQN), namely Graphic Convolution Q network (GCQ) is proposed as the information fusion module and decision processor. The proposed model can aggregate the information acquired from collaborative sensing and output safe and cooperative lane changing decisions for multiple CAVs so that individual intention can be satisfied even under a highly dynamic and partially observed mixed traffic. The proposed algorithm can be deployed on centralized control infrastructures such as road-side units (RSU) or cloud platforms to improve the CAV operation.

研究动机与目标

• 解决在具有动态变化的智能体数量和大规模联合行动空间的 CAV 网络中实现协作控制的需求。
• 开发一个决策处理器，将来自协作感知的信息融合，以实现安全、协作的机动。
• 实现集中部署（RSU 或云端），以在混合、部分观测的交通情景中改善 CAV 的运行。

提出的方法

• 引入 Graphic Convolution Q network (GCQ) 作为信息融合和决策模块。
• 将 Graphic Convolutional Neural Networks 与 Deep Q-Networks 相结合，以处理融合后的感知数据。
• 处理高度动态和部分观测的交通情景，以产生协作的变道决策。
• 提供适用于路侧单元或云部署的集中控制框架。

实验结果

研究问题

• RQ1 如何有效融合协作感知数据，以支持多 CAV 的协作决策？
• RQ2 基于 GCQ 的 DRL 框架能否随着代理数量的变化和部分观测环境扩展？
• RQ3 在动态交通和集中控制下，可以达到的变道决策质量和安全性特性有哪些？

主要发现

• GCQ 从协作感知中聚合信息，产生安全、协作的变道决策。
• 该框架旨在在高度动态、部分观测的交通状况下满足单辆车的意图。
• 该方法设计用于在集中基础设施（如 RSU 或云平台）上部署，以改善 CAV 的运行。