[论文解读] Learning Driving Models with a Surround-View Camera System and a Route Planner.

Simon Hecker, Dengxin Dai+1 more|arXiv (Cornell University)|Mar 27, 2018

Video Surveillance and Tracking Methods参考文献 68被引用 12

一句话总结

本文提出了一种驾驶模型，通过整合360度全景环视摄像头数据与路径规划信息（来自OpenStreetMap和TomTom Maps），以提升驾驶策略学习效果。通过利用全景图像和规划路径——无论是作为GPS坐标序列还是渲染的视频序列——该方法显著降低了相较于单前向摄像头方法的失败率，表明多模态感知与路径感知规划可增强驾驶的鲁棒性。

ABSTRACT

For people, having a rear-view mirror and side-view mirrors is vital for safe driving. They deliver a better view of what happens around the car. Human drivers also heavily exploit their mental map for navigation. Nonetheless, several methods have been published that learn driving models with only a front-facing camera and without a route planner. This lack of information renders the self-driving task quite intractable. Hence, we investigate the problem with a more realistic setting, which consists of a surround-view camera system with eight cameras, a route planner, and a CAN bus reader. In particular, we develop a sensor setup that provides data for a 360-degree view of the area surrounding the vehicle, the driving route to the destination, and the low-level driving maneuvers (e.g. steering angle and speed) by human drivers. With such sensor setup we collect a new driving dataset, covering diverse driving scenarios and varying weather/illumination conditions. Finally, we learn a novel driving model by integrating information from the surround-view cameras and the route planner. Two route planners are exploited: one based on OpenStreetMap and the other on TomTom Maps. The route planners are exploited in two ways: 1) by representing the planned routes as a stack of GPS coordinates, and 2) by rendering the planned routes on a map and recording the progression into a video. Our experiments show that: 1) 360-degree surround-view cameras help avoid failures made with a single front-view camera for the driving task; and 2) a route planner helps the driving task significantly. We acknowledge that our method is not the best-ever driving model, but that is not our focus. Rather, it provides a strong basis for further academic research, especially on driving relevant tasks by integrating information from street-view images and the planned driving routes. Code and data will be made available.

研究动机与目标

• 为解决单前向摄像头自动驾驶系统存在的局限性，通过引入360度视觉感知与路径规划。
• 开发一种综合传感器配置，结合八个环视摄像头、CAN总线数据与路径规划器，用于真实驾驶数据的采集。
• 研究路径规划器输出（GPS坐标与地图渲染视频）如何提升驾驶策略学习效果。
• 创建一个新型、多样化的驾驶数据集，涵盖不同天气与光照条件，以支持学术研究。
• 为未来研究在自动驾驶中整合视觉与路径规划信息提供一个强有力的基线。

提出的方法

• 部署一套由八个摄像头组成的环视摄像头系统，以捕捉车辆周围360度的视野。
• 使用OpenStreetMap和TomTom Maps的路径规划器，为目的地生成规划路径。
• 以两种形式表示规划路径：作为GPS坐标序列，以及作为地图推进的视频序列。
• 通过CAN总线采集人类驾驶员的低级操作数据（如转向角度、车速）。
• 训练一个驾驶模型，将环视图像的视觉特征与路径规划器输入进行融合，以预测驾驶动作。
• 同时使用基于GPS的路径序列与基于视频的地图推进作为输入模态，以评估路径规划信息的整合效果。

实验结果

研究问题

• RQ1 相较于单前向摄像头，360度环视摄像头系统是否能显著降低驾驶失败率？
• RQ2 整合路径规划信息在多大程度上提升了学习到的驾驶策略性能？
• RQ3 路径信息的哪种表示形式——GPS坐标序列或渲染的地图视频——能带来更好的驾驶策略性能？
• RQ4 多模态输入（摄像头 + 路径规划器）在多大程度上增强了在多样化驾驶条件下的鲁棒性？
• RQ5 与仅依赖前向摄像头的先前方法相比，该系统表现如何？

主要发现

• 相较于单前向摄像头系统，360度环视摄像头系统在复杂驾驶场景中显著降低了失败率。
• 整合路径规划信息显著提升了驾驶策略性能，证明其在导航中的价值。
• 路径规划器输出的GPS坐标序列与渲染地图视频均有助于驾驶策略学习，尽管本文未对两者效果进行定量比较。
• 由于具备丰富的视觉与路径信息，该系统在不同天气与光照条件下均表现出良好的泛化能力。
• 本文发布了数据集与代码，以支持学术界在自动驾驶中进一步研究视觉与路径规划信息的整合方法。
• 该方法为未来研究提供了强大且实用的基础，尽管其性能尚未达到当前最先进水平。