[Paper Review] Data-Driven Vehicle Trajectory Forecasting

Shayan Jawed, Eya Boumaiza+2 more|arXiv (Cornell University)|Feb 9, 2019

Autonomous Vehicle Technology and Safety27 references4 citations

TL;DR

This paper proposes a convolutional neural network (CNN)-based approach for multi-step vehicle trajectory forecasting using raw camera data, framing the task as a time series regression problem. The method outperforms baselines like linear regression, decision trees, and mean/last-value predictions, achieving a mean squared error (MSE) of 3393.64 using a joint forecasting strategy, demonstrating superior accuracy and real-time inference capability for autonomous driving safety.

ABSTRACT

An active area of research is to increase the safety of self-driving vehicles. Although safety cannot be guarenteed completely, the capability of a vehicle to predict the future trajectories of its surrounding vehicles could help ensure this notion of safety to a greater deal. We cast the trajectory forecast problem in a multi-time step forecasting problem and develop a Convolutional Neural Network based approach to learn from trajectory sequences generated from completely raw dataset in real-time. Results show improvement over baselines.

Motivation & Objective

• To improve safety in autonomous driving by enabling accurate prediction of surrounding vehicles' future trajectories.
• To address the challenge of forecasting complex, noisy vehicle trajectories in dynamic traffic environments using real-time, raw camera data.
• To develop a computationally efficient, real-time inference system for trajectory forecasting without relying on expensive sensors like LiDAR.
• To compare the performance of independent vs. joint forecasting strategies in multi-step trajectory prediction using CNNs.
• To evaluate the proposed CNN framework against classical baselines and state-of-the-art regression models.

Proposed method

• The system uses state-of-the-art object detection and tracking to generate trajectory sequences from raw camera input, eliminating the need for LiDAR.
• A 1D convolutional neural network (CNN) is trained to predict future (x,y) positions of tracked vehicles over multiple timesteps.
• Two training strategies are evaluated: independent forecasting (five separate models per horizon) and joint forecasting (one model predicting all horizons simultaneously).
• The joint strategy is adopted for final evaluation due to its faster inference speed, critical for real-time applications.
• The model is trained end-to-end using Adam optimization with a fixed initial learning rate of 0.0001 and a batch size of 3.
• Loss is computed as mean squared error (MSE) between predicted and ground-truth trajectory points across multiple future timesteps.

Experimental results

Research questions

• RQ1 Can a CNN-based architecture outperform classical regression models like linear regression and decision trees in multi-step vehicle trajectory forecasting?
• RQ2 How does the joint forecasting strategy compare to independent forecasting in terms of prediction accuracy and inference speed?
• RQ3 Does the proposed method achieve real-time inference performance suitable for deployment in autonomous vehicles?
• RQ4 To what extent does the model's performance degrade over longer forecasting horizons?
• RQ5 Can a CNN effectively learn complex, non-linear trajectory patterns from raw trajectory sequences without relying on LiDAR or complex sensor fusion?

Key findings

• The joint CNN strategy achieved the lowest mean squared error (MSE) of 3393.64, significantly outperforming the next best baseline, the independent CNN approach, which had an MSE of 15181.55.
• The joint strategy reduced inference time compared to the independent approach, making it more suitable for real-time deployment in autonomous vehicles.
• The model's performance degraded with increasing forecast horizon, as evidenced by higher MSE values for later timesteps (e.g., 6192.45 for t₄₅₋₅₀), which is expected due to error accumulation.
• The baseline of predicting the last observed position had an MSE of 23511.61, while the mean prediction baseline had an MSE of 7259.255, showing that even simple baselines are challenging to beat.
• The joint strategy achieved a 79.4% reduction in MSE compared to the mean baseline and a 84.5% reduction compared to the last-value baseline, indicating strong predictive capability.
• The unsupervised tracking failure detection method showed high agreement (580 failures) with human evaluation (553 failures), validating its effectiveness for data quality control.