[Paper Review] YOLOPv2: Better, Faster, Stronger for Panoptic Driving Perception

Cheng Han, Qichao Zhao+4 more|arXiv (Cornell University)|Aug 24, 2022

Advanced Neural Network Applications52 citations

TL;DR

YOLOPv2 presents a multi-task network that jointly performs traffic object detection, drivable area segmentation, and lane detection, achieving state-of-the-art speed and accuracy on BDD100K (91 FPS on V100) with strong multi-task metrics.

ABSTRACT

Over the last decade, multi-tasking learning approaches have achieved promising results in solving panoptic driving perception problems, providing both high-precision and high-efficiency performance. It has become a popular paradigm when designing networks for real-time practical autonomous driving system, where computation resources are limited. This paper proposed an effective and efficient multi-task learning network to simultaneously perform the task of traffic object detection, drivable road area segmentation and lane detection. Our model achieved the new state-of-the-art (SOTA) performance in terms of accuracy and speed on the challenging BDD100K dataset. Especially, the inference time is reduced by half compared to the previous SOTA model. Code will be released in the near future.

Motivation & Objective

• Motivate real-time panoptic driving perception under limited compute resources.
• Develop an end-to-end multi-task network with a shared encoder and task-specific heads.
• Improve detection, drivable area segmentation, and lane detection accuracy while maintaining high inference speed.

Proposed method

• Use a shared encoder with three separate decoder heads for the three tasks.
• Adopt E-ELAN group convolution, SPP, and FPN for effective multi-scale feature fusion.
• Employ three task-specific heads with distinct connections to FPN/neck to optimize each task (drivable area prior to FPN; lanes at deeper FPN level).
• Maintain an anchor-based multi-scale detection head with PAN+FPN fusion for object detection.
• Use a hybrid loss combining Dice loss and focal loss for lane/drive tasks and cross-entropy/focal losses for detection, with Mosaic and Mixup augmentations to boost multi-task performance.
• Train with Cosine Annealing with warm restarts, specific hyperparameters, and image resizing (640x640 during training).

Experimental results

Research questions

• RQ1 Can a single, efficient multi-task network simultaneously achieve high-accuracy object detection, drivable area segmentation, and lane detection for driving scenes?
• RQ2 Does separating the decoder heads by task and using task-specific feature levels improve panoptic driving perception performance without sacrificing speed?
• RQ3 What is the impact of data augmentation (Mosaic, Mixup) and hybrid loss on the three tasks in a multi-task driving perception setting?

Key findings

• Achieves 0.83 mAP50 for traffic object detection on BDD100K.
• Achieves 0.93 mIoU for drivable area segmentation.
• Achieves 87.31% accuracy for lane detection.
• Runs at 91 FPS on NVIDIA Tesla V100, faster than the previous SOTA (YOLOP at 49 FPS).
• Model has 38.9M parameters (larger but faster) and outperforms YOLOP and HybridNets in combined metrics.
• Ablation studies show improvements from Mosaic+Mixup, focal loss, and Dice+Focal hybrid loss.