[论文解读] Deep Learning Based Automatic Video Annotation Tool for Self-Driving Car

M. Nagarajan, K. Ganesan|arXiv (Cornell University)|Apr 19, 2019

Video Surveillance and Tracking Methods参考文献 14被引用 9

一句话总结

本文提出了一种基于深度学习的自动驾驶汽车自动视频标注工具，整合了YOLO和RetinaNet进行目标检测，ResNet-50和VGG-19进行分类，LaneNet进行车道线检测，以及DeepSort进行目标跟踪。该工具与人工标注相比达到了83%的准确率，并在GPU上将530帧视频的处理速度提升了1200倍，显著减少了标注时间，同时保持了高精度。

ABSTRACT

In a self-driving car, objection detection, object classification, lane detection and object tracking are considered to be the crucial modules. In recent times, using the real time video one wants to narrate the scene captured by the camera fitted in our vehicle. To effectively implement this task, deep learning techniques and automatic video annotation tools are widely used. In the present paper, we compare the various techniques that are available for each module and choose the best algorithm among them by using appropriate metrics. For object detection, YOLO and Retinanet-50 are considered and the best one is chosen based on mean Average Precision (mAP). For object classification, we consider VGG-19 and Resnet-50 and select the best algorithm based on low error rate and good accuracy. For lane detection, Udacity's 'Finding Lane Line' and deep learning based LaneNet algorithms are compared and the best one that can accurately identify the given lane is chosen for implementation. As far as object tracking is concerned, we compare Udacity's 'Object Detection and Tracking' algorithm and deep learning based Deep Sort algorithm. Based on the accuracy of tracking the same object in many frames and predicting the movement of objects, the best algorithm is chosen. Our automatic video annotation tool is found to be 83% accurate when compared with a human annotator. We considered a video with 530 frames each of resolution 1035 x 1800 pixels. At an average each frame had about 15 objects. Our annotation tool consumed 43 minutes in a CPU based system and 2.58 minutes in a mid-level GPU based system to process all four modules. But the same video took nearly 3060 minutes for one human annotator to narrate the scene in the given video. Thus we claim that our proposed automatic video annotation tool is reasonably fast (about 1200 times in a GPU system) and accurate.

研究动机与目标

• 开发一种用于自动驾驶汽车的自动视频标注系统，以减少对人工标注的依赖。
• 对比并选择用于目标检测、分类、车道线检测和目标跟踪的最优深度学习模型。
• 评估所提出工具在准确率和处理时间方面与人工标注的性能表现。
• 通过端到端的深度学习流水线实现实时视频场景描述。

提出的方法

• 系统使用YOLO和RetinaNet-50进行目标检测，基于平均精度均值（mAP）选择最佳模型。
• 对于目标分类，通过准确率和错误率评估VGG-19和ResNet-50，以选择最优模型。
• 车道线检测采用Udacity的“Finding Lane Line”和基于深度学习的LaneNet，后者因更高精度被选中。
• 目标跟踪对比Udacity的跟踪算法与DeepSort，基于跟踪一致性和运动预测精度选择DeepSort。
• 最终的标注流水线将表现最佳的模型整合为统一工具，按顺序处理视频帧。
• 性能评估基于一个包含530帧的视频（1035×1800像素），每帧约有15个目标，以人工标注的真值作为参考。

实验结果

研究问题

• RQ1 在自动驾驶汽车视频流中，哪种深度学习模型在目标检测任务中能获得最高的平均精度均值（mAP）？
• RQ2 在目标识别任务中，VGG-19与ResNet-50哪个分类网络的错误率最低且准确率最高？
• RQ3 基于深度学习的LaneNet在车道线检测精度上与传统方法（如Udacity的“Finding Lane Line”）相比表现如何？
• RQ4 在目标跟踪任务中，Udacity的算法与DeepSort哪个更能保持目标身份并准确预测运动轨迹？
• RQ5 自动化工具在显著缩短处理时间的同时，能在多大程度上达到人工标注的准确率水平？

主要发现

• 所提出的自动视频标注工具与人工标注真值相比，达到了83%的准确率。
• 在基于GPU的系统上，该工具仅用2.58分钟处理完全部530帧视频，而人工标注耗时3060分钟，实现了1200倍的速度提升。
• 基于CPU的系统处理相同视频耗时43分钟，表明其性能对硬件依赖较强。
• 对YOLO和RetinaNet-50进行目标检测评估，最佳模型基于mAP选出，但摘要中未明确指出具体获胜模型。
• ResNet-50在分类任务中表现优于VGG-19，因其错误率更低且准确率更高。
• LaneNet在车道线检测性能上显著优于Udacity的“Finding Lane Line”算法。