[论文解读] ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modeling

Quanyi Li, Zhenghao Peng+5 more|arXiv (Cornell University)|Jun 21, 2023

Autonomous Vehicle Technology and Safety被引用 19

一句话总结

ScenarioNet 将来自多个数据集的真实世界驾驶数据汇聚到一个统一的开源平台，用于大规模交通场景仿真与建模，推动数据驱动的场景生成、跨数据集 RL/IL 训练、多智能体学习，以及在 MetaDrive 中对 AD 堆栈的测试。

ABSTRACT

Large-scale driving datasets such as Waymo Open Dataset and nuScenes substantially accelerate autonomous driving research, especially for perception tasks such as 3D detection and trajectory forecasting. Since the driving logs in these datasets contain HD maps and detailed object annotations which accurately reflect the real-world complexity of traffic behaviors, we can harvest a massive number of complex traffic scenarios and recreate their digital twins in simulation. Compared to the hand-crafted scenarios often used in existing simulators, data-driven scenarios collected from the real world can facilitate many research opportunities in machine learning and autonomous driving. In this work, we present ScenarioNet, an open-source platform for large-scale traffic scenario modeling and simulation. ScenarioNet defines a unified scenario description format and collects a large-scale repository of real-world traffic scenarios from the heterogeneous data in various driving datasets including Waymo, nuScenes, Lyft L5, and nuPlan datasets. These scenarios can be further replayed and interacted with in multiple views from Bird-Eye-View layout to realistic 3D rendering in MetaDrive simulator. This provides a benchmark for evaluating the safety of autonomous driving stacks in simulation before their real-world deployment. We further demonstrate the strengths of ScenarioNet on large-scale scenario generation, imitation learning, and reinforcement learning in both single-agent and multi-agent settings. Code, demo videos, and website are available at https://metadriverse.github.io/scenarionet.

研究动机与目标

• 提供一个统一格式，将异构驾驶数据转换为可扩展的交通场景。
• 从多个数据集聚合真实世界场景，以实现跨数据集的训练与评估。
• 在单智能体和多智能体设置中实现大规模场景生成、模仿学习与强化学习。
• 通过 ROS 桥接集成，支持基于仿真的自动驾驶堆栈测试。

提出的方法

• 定义一个具有四个顶层键的统一嵌套字典场景描述。
• 将 Waymo、nuScenes、nuPlan、L5、Argoverse 及程序化生成的数据转换为统一格式。
• 在 MetaDrive 仿真器中以 2D BEV 与 3D 渲染重放场景，并进行传感器仿真。
• 提供数据集管理操作（转换、完整性检查、合并、过滤、分割、采样），无拷贝且可并行化。
• 提供通过转换器添加新数据集的工具，并通过多进程扩展转换规模。
• 描述系统体系结构，数据层、系统层、应用层通过数据转换与仿真流连接。

Figure 1: Snapshots of three scenarios extracted from nuScenes and their corresponding interactive environments with multiple views and sensors including RGB camera, depth camera, and semantic camera. The RGB sensor can be used for end-to-end driving systems like Openpilot [ 10 ] .

实验结果

研究问题

• RQ1 真实世界驾驶数据集是否能够统一为可扩展、可互操作的仿真场景数据库？
• RQ2 跨数据集训练是否比单一数据集训练在模仿学习和强化学习中提升策略学习？
• RQ3 ScenarioNet 是否能够通过统一场景支持有效的多智能体学习与 AD 堆栈测试？
• RQ4 聚合合成场景与真实世界场景对策略泛化和仿真到现实的迁移有何影响？

主要发现

数据集	轨迹长度	车辆数量	行人数量	交叉口比例	构建比例
Waymo	136.55( ± 95.98)	89.93( ± 64.51)	11.7( ± 22.97)	0.71	0.0
nuPlan	95.48( ± 42.32)	53.96( ± 25.35)	21.99( ± 19.9)	0.57	1.0
PG	226.07( ± 70.7)	9.81( ± 3.31)	0.0	0.36	0.39

• 统一的场景描述 enabling 在 MetaDrive 中对多个数据集（Waymo、nuScenes、L5、nuPlan、Argoverse）的重放和交互。
• 跨数据集课程化 RL 实验表明，当在真实世界测试集上评估时，真实世界数据相较仅合成数据有更高的成功率。
• 将合成场景与真实世界场景结合时，在 nuPlan 测试中能获得更高的策略性能和对曲线轨迹的更好泛化。"
• 多智能体强化学习/模仿学习实验表明，在奖励、终止条件和评估中使用真实轨迹可实现有效学习。
• Openpilot 可以在 ScenarioNet 重构的真实世界场景中运行，展示端到端自动驾驶堆栈的实际测试。
• 表 2 与表 3 分别总结数据集统计及多智能体学习结果，显示数据集的轨道长度、车辆数量和 RL/IL 性能。

Figure 3: From bottom to top, ScenarioNet platform consists of the data layer, system layer, and application layer which are connected by two critical data flows, data conversion ( $\rightarrow$ ) and simulation ( $\rightarrow$ ) . Data conversing unifies various data formats and stores them in an i