[Paper Review] Multi-Fidelity Recursive Behavior Prediction

Mihir Jain, Kyle Brown+1 more|arXiv (Cornell University)|Dec 18, 2018

Autonomous Vehicle Technology and Safety19 references4 citations

TL;DR

This paper proposes Multi-Fidelity Recursive Behavior Prediction (MFRBP), a game-theoretic model that improves vehicle trajectory prediction accuracy by recursively reasoning about future interactions among agents using Level-$k$ and Cognitive Hierarchy paradigms. It achieves state-of-the-art performance on the NGSIM dataset, reducing root mean square error (RMSE) compared to prior methods, especially over longer prediction horizons, through multi-fidelity modeling that assigns higher reasoning levels to key agents like the ego vehicle.

ABSTRACT

Predicting the behavior of surrounding vehicles is a critical problem in automated driving. We present a novel game theoretic behavior prediction model that achieves state of the art prediction accuracy by explicitly reasoning about possible future interaction between agents. We evaluate our approach on the NGSIM vehicle trajectory data set and demonstrate lower root mean square error than state-of-the-art methods.

Motivation & Objective

• To improve the accuracy of vehicle behavior prediction in autonomous driving by explicitly modeling future interactions between traffic agents.
• To address the limitation of existing models that either ignore interactions or implicitly condition on scene context, by introducing a recursive, game-theoretic reasoning framework.
• To enable multi-fidelity prediction, where different agents are modeled with varying levels of reasoning fidelity based on their importance (e.g., ego vehicle vs. peripheral vehicles).
• To evaluate the impact of conditioning predictions on a planned ego trajectory, simulating real-world automated driving scenarios.
• To demonstrate that recursive, multi-fidelity reasoning leads to measurable improvements in prediction accuracy over state-of-the-art baselines.

Proposed method

• The model uses a recursive prediction scheme inspired by Level-$k$ and Cognitive Hierarchy reasoning, where each agent’s trajectory at level $k$ depends on the highest-level predictions of other agents up to $k-1$.
• Each agent is assigned a reasoning level $k_i$ and a sequence of policy models $(\pi_{i,0}, \dots, \pi_{i,k_i})$, with higher levels incorporating more sophisticated interaction modeling.
• Trajectory predictions are modeled as Gaussian distributions with mean $\hat{\mathbf{x}}_{i,k}^{(t+1:t_f)}$ and covariance $\Sigma_{i,k}$, conditioned on the current history and prior-level predictions of other agents.
• Multi-fidelity modeling is implemented by assigning lower reasoning levels ($k_i = 0$) and simpler policy models (e.g., constant velocity) to peripheral agents, while ego agents use higher levels and more complex models.
• The model integrates components from Convolutional Social Pooling (CSP) for feature extraction and trajectory prediction, using leaky ReLU and Adam optimization in baseline implementations.
• Experiments include three variants: L1-RBP (single-level), L1-MFRBP (multi-fidelity ego-centric), and a planning-aware variant where ground-truth ego trajectories are used as input to condition predictions.

Experimental results

Research questions

• RQ1 Can recursive game-theoretic reasoning about future agent interactions improve trajectory prediction accuracy compared to non-interactive or implicitly interactive models?
• RQ2 Does multi-fidelity modeling—assigning different reasoning levels and policy models to different agents—enhance prediction performance, especially in ego-centric autonomous driving scenarios?
• RQ3 How much does conditioning the prediction on a candidate future ego trajectory (e.g., a planned path) improve accuracy?
• RQ4 Can the proposed recursive framework outperform state-of-the-art baselines like CSP on standard benchmarks such as NGSIM?
• RQ5 What is the impact of increasing reasoning depth (higher $k$) on prediction error, particularly over longer time horizons?

Key findings

• L1-RBP achieved lower RMSE than the baseline $\mathbf{\pi}_{\text{CSP}}$ model across all time horizons, with a 14.5% relative reduction in RMSE at 1 second and 8.5% at 5 seconds.
• L1-MFRBP, which applies multi-fidelity modeling with ego-centric focus, showed slight but consistent improvement over the baseline, with 1.20 RMSE at 2 seconds and 4.16 RMSE at 5 seconds.
• The planning-aware variant of L1-MFRBP, which conditions on ground-truth ego trajectories, achieved the best results, with 2.88 RMSE at 4 seconds and 4.01 RMSE at 5 seconds, indicating that access to planned trajectories enhances prediction accuracy.
• Improvements were more pronounced over longer prediction horizons, suggesting that recursive reasoning about future interactions is particularly beneficial for long-term motion forecasting.
• The model’s recursive structure enables scalable, hierarchical reasoning about interactions without requiring full joint trajectory optimization.
• The integration of CSP-based features with recursive game-theoretic reasoning demonstrates the effectiveness of combining deep learning with structured reasoning in behavior prediction.