[论文解读] A Memory-Augmented Multi-Task Collaborative Framework for Unsupervised Traffic Accident Detection in Driving Videos

Rongqin Liang, Yuanman Li+3 more|arXiv (Cornell University)|Jul 27, 2023

Autonomous Vehicle Technology and SafetyEngineering被引用 3

一句话总结

该论文提出了一种用于驾驶视频中无监督交通碰撞检测的记忆增强多任务协作框架（MAMTCF），通过联合利用光流重建与未来目标定位，检测自身车辆参与及非自身车辆参与的碰撞事件。通过整合一种记忆增强运动表征机制，将存储的正常模式融入运动特征，该方法在大规模基准数据集上实现了最先进性能，优于单一预训练任务方法。

ABSTRACT

Identifying traffic accidents in driving videos is crucial to ensuring the safety of autonomous driving and driver assistance systems. To address the potential danger caused by the long-tailed distribution of driving events, existing traffic accident detection (TAD) methods mainly rely on unsupervised learning. However, TAD is still challenging due to the rapid movement of cameras and dynamic scenes in driving scenarios. Existing unsupervised TAD methods mainly rely on a single pretext task, i.e., an appearance-based or future object localization task, to detect accidents. However, appearance-based approaches are easily disturbed by the rapid movement of the camera and changes in illumination, which significantly reduce the performance of traffic accident detection. Methods based on future object localization may fail to capture appearance changes in video frames, making it difficult to detect ego-involved accidents (e.g., out of control of the ego-vehicle). In this paper, we propose a novel memory-augmented multi-task collaborative framework (MAMTCF) for unsupervised traffic accident detection in driving videos. Different from previous approaches, our method can more accurately detect both ego-involved and non-ego accidents by simultaneously modeling appearance changes and object motions in video frames through the collaboration of optical flow reconstruction and future object localization tasks. Further, we introduce a memory-augmented motion representation mechanism to fully explore the interrelation between different types of motion representations and exploit the high-level features of normal traffic patterns stored in memory to augment motion representations, thus enlarging the difference from anomalies. Experimental results on recently published large-scale dataset demonstrate that our method achieves better performance compared to previous state-of-the-art approaches.

研究动机与目标

• 解决驾驶视频中无监督交通碰撞检测的挑战，尤其针对碰撞类型分布长尾的问题。
• 克服单一预训练任务方法（特别是基于外观与未来目标定位的方法）的局限性，这些方法因对相机运动与光照变化敏感，难以检测自身车辆参与的碰撞。
• 通过联合建模外观变化（通过光流）与目标运动（通过未来定位），提升检测灵敏度，捕捉自身运动与目标级异常。
• 通过从记忆库中引入高层正常交通模式，增强运动表征学习，提高异常判别能力。
• 开发一种记忆增强运动表征机制，动态选择相关记忆项以优化运动特征，提升模型鲁棒性与泛化能力。

提出的方法

• 提出一种多任务协作框架，联合优化光流重建与未来目标定位作为预训练任务，以建模正常驾驶模式。
• 引入一种记忆增强运动表征（MAMR）机制，利用可学习的记忆库存储正常交通的高层特征，融合来自两项任务的运动表征。
• 采用具有可学习阈值的硬剪枝操作，强制记忆访问的稀疏性，仅选择最相关的记忆项用于运动表征优化。
• 采用基于Transformer的架构，聚合并优化时序帧间的运动表征，提升动态场景的时序建模能力。
• 采用记忆更新策略，定期用来自正常交通序列的新特征替换过时的记忆项，保持对正常行为的最新表征。
• 以端到端无监督方式训练模型，利用重建损失与定位损失引导特征学习，无需事故标注。

Figure 1: The framework of our MAMTCF algorithm. MAMTCF primarily consists of a feature extraction module, a memory-augmented motion representation mechanism, and a multi-task decoder. 1) First, the optical flow of driving video frames and the bounding boxes of observed objects in the scenes are enc

实验结果

研究问题

• RQ1 与单任务基线相比，联合优化光流重建与未来目标定位是否能提升对自身车辆参与及非自身车辆参与碰撞的检测性能？
• RQ2 引入记忆增强机制如何提升无监督TAD中的运动表征学习与异常判别能力？
• RQ3 记忆大小与硬剪枝阈值的最优配置为何种组合，可在MAMR机制中实现性能与泛化能力的平衡？
• RQ4 该框架在大规模真实驾驶视频基准上，相较于现有无监督TAD方法，性能提升程度如何？
• RQ5 为何该方法在自身车辆运动微弱或光照极端变化的场景中仍会失效？其关键失败模式是什么？

主要发现

• 所提出的MAMTCF框架在近期发布的大型驾驶视频数据集上实现了最先进性能，无论在mAP还是F1-score上均优于现有无监督TAD方法。
• 记忆增强运动表征机制通过增大正常与异常运动模式之间的判别边界，提升了异常检测能力，尤其在自身车辆参与的碰撞检测中表现显著。
• 当记忆槽数量M=1000时性能最佳，超过此值性能略有下降，原因在于过大的记忆容量导致过拟合。
• 采用λ=3/M的硬剪枝操作获得最佳性能，优于使用Softmax的基线方法，因其促进稀疏性并选择更相关的记忆项。
• 在MAMR机制中，L=3层的模型实现性能与模型复杂度的最佳平衡，更深网络反而轻微降低检测准确率。
• 失败案例揭示了在自身车辆运动微弱或能见度低的场景中检测能力受限，原因在于光流变化微弱且目标检测失败。

Figure 2: Illustration of the propose MAMR mechanism. The MAMR mechanism mainly consists of a inter-motion layer, memory-augmented motion layer, and feedforward network. The inter-motion layer explores the interrelation between global motion and object motion , while the memory-augmented motion laye