[Paper Review] Interpretable Self-Aware Neural Networks for Robust Trajectory Prediction

Masha Itkina, Mykel J. Kochenderfer|arXiv (Cornell University)|Nov 16, 2022

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper proposes Interpretable Self-Aware Prediction (ISAP), a one-shot, single-forward-pass trajectory prediction framework using evidential deep learning to estimate epistemic uncertainty across interpretable semantic concepts—agent behavior, road structure, and social context. It outperforms state-of-the-art baselines in uncertainty estimation while maintaining strong prediction accuracy, even without OOD data during training.

ABSTRACT

Although neural networks have seen tremendous success as predictive models in a variety of domains, they can be overly confident in their predictions on out-of-distribution (OOD) data. To be viable for safety-critical applications, like autonomous vehicles, neural networks must accurately estimate their epistemic or model uncertainty, achieving a level of system self-awareness. Techniques for epistemic uncertainty quantification often require OOD data during training or multiple neural network forward passes during inference. These approaches may not be suitable for real-time performance on high-dimensional inputs. Furthermore, existing methods lack interpretability of the estimated uncertainty, which limits their usefulness both to engineers for further system development and to downstream modules in the autonomy stack. We propose the use of evidential deep learning to estimate the epistemic uncertainty over a low-dimensional, interpretable latent space in a trajectory prediction setting. We introduce an interpretable paradigm for trajectory prediction that distributes the uncertainty among the semantic concepts: past agent behavior, road structure, and social context. We validate our approach on real-world autonomous driving data, demonstrating superior performance over state-of-the-art baselines. Our code is available at: https://github.com/sisl/InterpretableSelfAwarePrediction.

Motivation & Objective

• Address the lack of model self-awareness in neural networks, particularly overconfidence in out-of-distribution (OOD) scenarios common in autonomous vehicle perception.
• Overcome limitations of existing uncertainty estimation methods that require multiple forward passes or OOD data during training.
• Improve interpretability of uncertainty by distributing it across semantically meaningful latent factors: agent behavior, map structure, and social context.
• Enable real-time deployment in safety-critical systems by ensuring single-inference pass performance.
• Provide engineers and downstream autonomy modules with actionable insights into the sources of uncertainty

Proposed method

• Employ evidential deep learning to model epistemic uncertainty via Dirichlet distributions over latent representations of agent behavior, HD maps, and social context.
• Use normalizing flows to constrain Dirichlet parameters during training, ensuring proper calibration without requiring OOD data.
• Decompose the input into three modality-specific encoders: agent trajectory, map embedding, and social context representation.
• Learn separate uncertainty parameters α_agent, α_map, and α_sc for each semantic component, enabling interpretable uncertainty decomposition.
• Combine the individual Dirichlet distributions into a final output distribution for trajectory prediction, with the sum α₀ serving as a global epistemic uncertainty indicator.
• Train the model end-to-end using a likelihood-based objective, with uncertainty estimation performed in a single forward pass

Experimental results

Research questions

• RQ1 Can epistemic uncertainty be effectively estimated in a single forward pass without requiring OOD data during training?
• RQ2 Can uncertainty be meaningfully decomposed across interpretable semantic factors such as agent behavior, road structure, and social context?
• RQ3 Does the proposed interpretable uncertainty decomposition improve the interpretability and diagnostic value of uncertainty estimates for engineers and downstream systems?
• RQ4 How does the model’s uncertainty estimation performance compare to state-of-the-art methods like Monte Carlo dropout and ensembles in both in-distribution and OOD settings?
• RQ5 Can the model detect and appropriately respond to distribution shift by reflecting it in the uncertainty of specific semantic components?

Key findings

• The ISAP framework achieves superior epistemic uncertainty estimation compared to state-of-the-art baselines such as MC dropout and ensembles, particularly in detecting OOD inputs.
• In the past trajectory experiment, OOD inputs with faster trajectories showed significantly lower α₀,agent values (69 vs. 8137) than ID inputs, correctly indicating higher uncertainty.
• In the map-based experiment, OOD inputs featuring roundabouts—unseen in training—produced a much lower α₀,map value (117 vs. 1060) than ID inputs, demonstrating correct detection of distribution shift.
• The social context uncertainty component (α₀,sc) remained high in both ID and OOD cases involving dense traffic, indicating that such scenarios are common in training data and thus not surprising.
• The model’s uncertainty decomposition provides interpretable insights: for example, low α₀,map in OOD roundabout scenarios clearly points to map structure as the source of uncertainty.
• Despite adding uncertainty estimation, the trajectory prediction performance of ISAP remains comparable to the baseline model, with only a minor drop in metrics