[论文解读] Driver Action Prediction Using Deep (Bidirectional) Recurrent Neural Network

Oluwatobi Olabiyi, Eric Martinson+2 more|arXiv (Cornell University)|Jun 7, 2017

Human Pose and Action Recognition参考文献 26被引用 71

一句话总结

本文将驾驶员行动预测表述为一个时间序列异常检测问题，并使用一个深度双向循环神经网络在执行前最多5秒，通过融合摄像头/环境、驾驶员和车辆动力学数据来预测行动（加速、制动、变道、转弯）。

ABSTRACT

Advanced driver assistance systems (ADAS) can be significantly improved with effective driver action prediction (DAP). Predicting driver actions early and accurately can help mitigate the effects of potentially unsafe driving behaviors and avoid possible accidents. In this paper, we formulate driver action prediction as a timeseries anomaly prediction problem. While the anomaly (driver actions of interest) detection might be trivial in this context, finding patterns that consistently precede an anomaly requires searching for or extracting features across multi-modal sensory inputs. We present such a driver action prediction system, including a real-time data acquisition, processing and learning framework for predicting future or impending driver action. The proposed system incorporates camera-based knowledge of the driving environment and the driver themselves, in addition to traditional vehicle dynamics. It then uses a deep bidirectional recurrent neural network (DBRNN) to learn the correlation between sensory inputs and impending driver behavior achieving accurate and high horizon action prediction. The proposed system performs better than other existing systems on driver action prediction tasks and can accurately predict key driver actions including acceleration, braking, lane change and turning at durations of 5sec before the action is executed by the driver.

研究动机与目标

• 通过及早且准确的驾驶员行动预测来提升ADAS。
• 利用多模态感知输入（环境、驾驶员、车辆动态）来实现预判。
• 将驾驶员行动预测表述为一个时间序列异常检测问题。
• 开发一个实时数据获取、处理与学习框架。

提出的方法

• 应用一个深度双向循环神经网络（DBRNN）来学习感知输入与即将出现的驾驶员行为之间的相关性。
• 将基于摄像头的驾驶环境数据与驾驶员信号和传统车辆动力学相结合。
• 将即将发生的驾驶员行动视为需要在时间序列框架内预测的异常。
• 实现一个用于持续预测的实时数据获取、处理和学习管线。
• 在关键动作上对系统进行评估——加速、制动、变道和转弯，预测时间步长为5秒。

实验结果

研究问题

• RQ1 DBRNN 能否有效学习多模态输入与即将发生的驾驶员行动之间的相关性？
• RQ2 使用所提框架，驾驶员行动能够多早且多准确地被预测？
• RQ3 与现有系统相比，DBRNN 方法是否在加速、制动、变道和转弯的预测上有所改进？
• RQ4 将驾驶员行动预测表述为时间序列异常检测问题的优点和局限性是什么？

主要发现

• 基于 DBRNN 的系统学习感知输入与即将发生的驾驶员行为之间的相关性。
• 该框架能够实时运行，整合摄像头、驾驶员和车辆动力学数据。
• 所提出的方法在驾驶员行动预测任务上优于现有系统。
• 它能够在执行前5秒的时间窗内准确预测关键动作（加速、制动、变道、转弯）。