[Paper Review] TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors

Simon Su, Sebastian Regalado+2 more|arXiv (Cornell University)|Jan 17, 2021

Traffic control and management37 references4 citations

TL;DR

TrafficSim proposes a differentiable, latent-variable-based multi-agent motion policy that learns from real-world driving data to generate diverse, socially consistent traffic behaviors. By unrolling the policy during training and optimizing through a fully differentiable simulation with a time-adaptive multi-task loss, it achieves state-of-the-art realism and reduces collision rates by 10% compared to baselines, enabling effective data augmentation for motion planners.

ABSTRACT

Simulation has the potential to massively scale evaluation of self-driving systems enabling rapid development as well as safe deployment. To close the gap between simulation and the real world, we need to simulate realistic multi-agent behaviors. Existing simulation environments rely on heuristic-based models that directly encode traffic rules, which cannot capture irregular maneuvers (e.g., nudging, U-turns) and complex interactions (e.g., yielding, merging). In contrast, we leverage real-world data to learn directly from human demonstration and thus capture a more diverse set of actor behaviors. To this end, we propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy that generates socially-consistent plans for all actors in the scene jointly. To learn a robust policy amenable for long horizon simulation, we unroll the policy in training and optimize through the fully differentiable simulation across time. Our learning objective incorporates both human demonstrations as well as common sense. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines. Notably, we can exploit trajectories generated by TrafficSim as effective data augmentation for training better motion planner.

Motivation & Objective

• To close the gap between simulated and real-world traffic behaviors by learning from human driving demonstrations rather than heuristic rules.
• To generate diverse, socially consistent multi-agent trajectories that include complex interactions like yielding, merging, and irregular maneuvers.
• To enable long-horizon simulation with robustness to distributional shift and minimal collision rates.
• To support interactive scenario design and data augmentation for self-driving motion planners.
• To balance imitation of human behavior with common-sense constraints through a time-adaptive loss function.

Proposed method

• Uses an implicit latent variable model to parameterize a joint multi-agent policy that generates plans for all actors simultaneously.
• Employs back-propagation through a fully differentiable simulation to optimize the policy end-to-end using real trajectory demonstrations.
• Unrolls the policy during training to mitigate error compounding and improve long-horizon consistency.
• Introduces a time-adaptive multi-task loss that balances imitation of demonstrated trajectories and adherence to common-sense traffic rules at each timestep.
• Models future actor plans over a horizon $T_{\text{plan}}$, enabling multi-step inference and efficient simulation with up to 4x speedup.
• Supports runtime optimization via rejection sampling or gradient-based refinement of latent variables to enforce collision avoidance and rule compliance.

Figure 1: Generating realistic multi-agent behaviors is a key component for simulation

Experimental results

Research questions

• RQ1 Can a learning-based multi-agent motion policy trained on real-world data generate more realistic and diverse traffic behaviors than heuristic rule-based models?
• RQ2 How can a joint multi-agent policy be trained effectively over long horizons without compounding error?
• RQ3 To what extent can trajectories generated by a learned simulator improve motion planner performance through data augmentation?
• RQ4 Can the simulation be accelerated via multi-step updates without sacrificing realism or safety?
• RQ5 How effective are runtime optimization techniques in reducing collisions while preserving behavioral diversity?

Key findings

• TrafficSim achieves the lowest scenario reconstruction error among all baselines, including heuristic, motion forecasting, and imitation learning models.
• The collision rate in TrafficSim-generated scenarios is 10.73%, significantly lower than heuristic models (e.g., 22.05% for IDM) and competitive with state-of-the-art imitation models.
• A motion planner trained on synthetic data from TrafficSim achieves a 4.52m mean L2 error, outperforming a planner trained on real data (4.85m) and surpassing all baselines in progress metrics.
• Multi-step inference with $\kappa=4$ achieves up to 4x speedup in simulation runtime (0.24s at 0.5Hz) with minimal degradation in scenario quality.
• Runtime optimization via gradient-based refinement reduces collision rate to 0.12% (from 0.50% without) while maintaining realistic trajectory diversity.
• Ablation studies confirm that closed-loop training with back-propagation through simulation is essential for robust long-horizon behavior generation.

Figure 2: Complex human driving behavior observed in the real world: red is actor of interest, green are interacting actors