[论文解读] AccidentGPT: Accident Analysis and Prevention from V2X Environmental Perception with Multi-modal Large Model

Lening Wang, Yilong Ren+10 more|arXiv (Cornell University)|Dec 20, 2023

Autonomous Vehicle Technology and Safety被引用 15

一句话总结

AccidentGPT 引入一个多模态大模型，将V2X环境感知与基于GPT的推理融合，用于分析事故并在自主、有人驾驶和执法场景中帮助预防。

ABSTRACT

Traffic accidents, being a significant contributor to both human casualties and property damage, have long been a focal point of research for many scholars in the field of traffic safety. However, previous studies, whether focusing on static environmental assessments or dynamic driving analyses, as well as pre-accident predictions or post-accident rule analyses, have typically been conducted in isolation. There has been a lack of an effective framework for developing a comprehensive understanding and application of traffic safety. To address this gap, this paper introduces AccidentGPT, a comprehensive accident analysis and prevention multi-modal large model. AccidentGPT establishes a multi-modal information interaction framework grounded in multi-sensor perception, thereby enabling a holistic approach to accident analysis and prevention in the field of traffic safety. Specifically, our capabilities can be categorized as follows: for autonomous driving vehicles, we provide comprehensive environmental perception and understanding to control the vehicle and avoid collisions. For human-driven vehicles, we offer proactive long-range safety warnings and blind-spot alerts while also providing safety driving recommendations and behavioral norms through human-machine dialogue and interaction. Additionally, for traffic police and management agencies, our framework supports intelligent and real-time analysis of traffic safety, encompassing pedestrian, vehicles, roads, and the environment through collaborative perception from multiple vehicles and road testing devices. The system is also capable of providing a thorough analysis of accident causes and liability after vehicle collisions. Our framework stands as the first large model to integrate comprehensive scene understanding into traffic safety studies. Project page: https://accidentgpt.github.io

研究动机与目标

• 通过整合感知与推理，推动对静态环境、动态状态、以及事前/事后任务的全面交通安全分析。
• 开发一个统一框架，将多传感器V2X感知与大语言模型融合，实现端到端的事故分析与预防。
• 使自动驾驶车辆能够感知与预测以避免碰撞，同时为人车驾驶提供主动警告和安全指南。
• 通过实时、协作感知和全面的事故因果分析，支持交通警察与管理。
• 展示首个提供交通安全研究全面场景理解的大模型集成。

提出的方法

• 创建一个V2X感知模块，将来自多辆车和道路设备的全景图像融合，生成3D检测、BEV地图和轨迹预测。
• 实现一个基于GPT-4V的推理模块，采用多层次优先抽样提示系统、主动/被动任务提示，以及用于事故相关任务的专用输出模块。
• 通过Ego-motion对齐多时间序列感知数据，以实现BEV特征融合和多任务检测头。
• 开发一个基于语料库的提示管线，动态构建时间序列语料，检索历史与上下文数据，并支持主动与被动的驾驶员提示。
• 在LLM计算中融入连锁思维风格的推理流程，以提升安全决策和不确定性处理。
• 从驾驶结果自动更新语料库，逐步提升推理和风险评估。

Figure 1: The overall framework of AccidentGPT.

实验结果

研究问题

• RQ1 将多车辆与车路协同感知与基于LLM的推理结合，对事故分析与预防的影响为何？
• RQ2 V2X感知输出（3D检测、BEV感知、轨迹）如何有效为GPT-4V推理提供 proactive 警告和事后因果分析？
• RQ3 主动/被动提示与迭代的语料更新能否在现实交通情景中改善安全建议和责任评估？

主要发现

• V2X感知（我们的方法）在表1中的指标高于列出的基线，检测率41.07%，mIOU57.3%，VPQ45.2%。
• 与其他模型相比，我们的方法在多个基线上提升了BEV感知和轨迹预测性能。
• 表2显示模型的Car/Truck/Van/Pedestrian的mATE、mASE和mAOE指标，Pedestrian行给出详细误差度量（mATE 1.0017，mASE 0.9993，mAOE 1.0253）。
• GPT推理模块通过人机对话实现主动的长程安全警告、盲区警报和针对驾驶员的安全建议。
• 该框架支持实时事故因果分析和交通管理机构的责任评估。

Figure 2: The V2X-perception architecture.