[Paper Review] Predicting Motion of Vulnerable Road Users using High-Definition Maps and Efficient ConvNets

Fang‐Chieh Chou, Tsung-Han Lin+7 more|arXiv (Cornell University)|Jun 20, 2019

Autonomous Vehicle Technology and Safety4 citations

TL;DR

This paper proposes a deep learning framework that uses high-definition map rasterization and efficient ConvNets to predict pedestrian and bicyclist motion in urban environments. By optimizing rasterization resolution, rotation, and scene context (e.g., traffic lights, lane directions), the method achieves high accuracy and low latency, enabling real-time deployment in self-driving vehicles.

ABSTRACT

Following detection and tracking of traffic actors, prediction of their future motion is the next critical component of a self-driving vehicle (SDV) technology, allowing the SDV to operate safely and efficiently in its environment. This is particularly important when it comes to vulnerable road users (VRUs), such as pedestrians and bicyclists. These actors need to be handled with special care due to an increased risk of injury, as well as the fact that their behavior is less predictable than that of motorized actors. To address this issue, in the current study we present a deep learning-based method for predicting VRU movement, where we rasterize high-definition maps and actor's surroundings into a bird's-eye view image used as an input to deep convolutional networks. In addition, we propose a fast architecture suitable for real-time inference, and perform an ablation study of various rasterization approaches to find the optimal choice for accurate prediction. The results strongly indicate benefits of using the proposed approach for motion prediction of VRUs, both in terms of accuracy and latency.

Motivation & Objective

• Address the challenge of predicting motion for vulnerable road users (VRUs), such as pedestrians and bicyclists, who are more prone to injury and exhibit less predictable behavior than motorized vehicles.
• Improve motion prediction accuracy by integrating detailed scene context—such as lane topology, traffic signals, and road geometry—into a deep learning framework.
• Develop a fast, efficient CNN architecture suitable for real-time inference on self-driving vehicles operating in dense urban environments.
• Conduct a comprehensive ablation study to identify optimal rasterization settings (resolution, rotation, feature encoding) that maximize prediction performance for VRUs.
• Validate the system through offline testing and successful onboard deployment in real-world SDV operations.

Proposed method

• Rasterize high-definition (HD) maps and surrounding scene context—including lane geometry, traffic signals, and actor positions—into bird’s-eye-view (BEV) feature maps at varying resolutions (0.1m, 0.2m, 0.3m).
• Rotate the rasterized BEV feature maps so that the actor’s heading points upward, reducing input variation and simplifying the prediction task for the CNN.
• Use a lightweight, efficient ConvNet architecture designed for real-time inference on embedded systems in self-driving vehicles.
• Incorporate scene-specific features such as traffic light states and lane direction indicators (encoded via color) into the raster input to model contextual constraints on VRU motion.
• Fine-tune the VRU prediction models using pre-trained vehicle motion models to leverage general motion patterns, while adapting to VRU-specific behaviors.
• Evaluate multiple rasterization configurations via ablation studies to isolate the impact of resolution, rotation, and feature encoding on prediction accuracy.

Experimental results

Research questions

• RQ1 How does the choice of rasterization resolution (0.1m, 0.2m, 0.3m) affect motion prediction accuracy for pedestrians and bicyclists?
• RQ2 To what extent does rotating the rasterized BEV map to align the actor’s heading with the top of the image improve prediction performance?
• RQ3 How critical is the inclusion of traffic light state and lane direction information in the rasterized input for accurate VRU trajectory prediction?
• RQ4 Can pre-trained vehicle motion models be effectively fine-tuned for VRU prediction, and how does this compare to training from scratch?
• RQ5 Which combination of rasterization settings yields the optimal balance between accuracy and computational efficiency for real-time SDV deployment?

Key findings

• A 0.1m resolution rasterization achieved the lowest prediction error for pedestrians, while 0.3m resolution performed worst, indicating that finer spatial detail benefits slower-moving actors.
• Not rotating the rasterized input led to a significant drop in accuracy for both pedestrians and bicyclists, demonstrating that rotation stabilizes the input distribution and simplifies learning.
• Excluding traffic light state information increased prediction error for both actor types, with a clear illustrative case showing that traffic light state changed a bicyclist’s predicted trajectory from crossing to waiting.
• Lane direction encoding improved bicyclist prediction accuracy slightly, but had no effect on pedestrians, confirming that lane guidance is more relevant for vehicle-like behavior.
• Learning raster colors end-to-end slightly improved bicyclist prediction but degraded pedestrian performance, suggesting that manually designed color encoding already captures sufficient signal for pedestrians.
• Fine-tuning with vehicle pre-trained weights improved bicyclist prediction over the baseline, but harmed pedestrian prediction—indicating that vehicle motion patterns are less transferable to pedestrians due to behavioral differences.