[Paper Review] Guided Conditional Diffusion for Controllable Traffic Simulation

Ziyuan Zhong, Davis Rempe+6 more|arXiv (Cornell University)|Oct 31, 2022

Traffic Prediction and Management Techniques4 citations

TL;DR

This paper proposes Controllable Traffic Generation (CTG), a conditional diffusion model that enables flexible, real-time control of traffic simulations using Signal Temporal Logic (STL) rules. By integrating differentiable STL robustness with diffusion model guidance, CTG generates physically feasible, realistic trajectories while satisfying user-defined objectives like goal reaching, speed limits, and collision avoidance, outperforming strong baselines in controllability and realism trade-offs on the nuScenes dataset.

ABSTRACT

Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles. Typical heuristic-based traffic models offer flexible control to make vehicles follow specific trajectories and traffic rules. On the other hand, data-driven approaches generate realistic and human-like behaviors, improving transfer from simulated to real-world traffic. However, to the best of our knowledge, no traffic model offers both controllability and realism. In this work, we develop a conditional diffusion model for controllable traffic generation (CTG) that allows users to control desired properties of trajectories at test time (e.g., reach a goal or follow a speed limit) while maintaining realism and physical feasibility through enforced dynamics. The key technical idea is to leverage recent advances from diffusion modeling and differentiable logic to guide generated trajectories to meet rules defined using signal temporal logic (STL). We further extend guidance to multi-agent settings and enable interaction-based rules like collision avoidance. CTG is extensively evaluated on the nuScenes dataset for diverse and composite rules, demonstrating improvement over strong baselines in terms of the controllability-realism tradeoff.

Motivation & Objective

• To address the gap between realism and controllability in autonomous vehicle simulation.
• To enable user-defined, test-time control of traffic behaviors (e.g., reaching goals, obeying speed limits) without retraining.
• To maintain physical feasibility and human-like behavior in generated trajectories under diverse, complex rules.
• To extend diffusion model guidance to multi-agent traffic scenarios with interaction-based constraints.
• To provide a scalable, differentiable framework for specifying and enforcing spatio-temporal driving rules using Signal Temporal Logic (STL).

Proposed method

• A conditional diffusion model is trained to generate agent-centric trajectories conditioned on local roadmaps and neighboring agents.
• Physical feasibility is enforced by modeling the state space using unicycle dynamics, ensuring realistic motion trajectories.
• Guidance is applied during sampling using the differentiable robustness of STL formulas to steer trajectories toward satisfying user-defined rules.
• A joint guidance procedure denoises all agents simultaneously to prevent interaction rule violations (e.g., collisions) in multi-agent scenes.
• Filtration and action trajectory optimization are used to refine samples, improving rule satisfaction and realism.
• The framework supports dynamic rule changes at inference time, enabling reactive simulation of AV scenarios.

Experimental results

Research questions

• RQ1 Can a diffusion-based traffic model achieve both high realism and flexible controllability under user-defined rules?
• RQ2 How effective is differentiable STL robustness in guiding trajectory generation for complex, spatio-temporal driving rules?
• RQ3 Can joint multi-agent guidance maintain interaction constraints (e.g., collision avoidance) while preserving realism?
• RQ4 How do design choices—such as dynamics enforcement, filtration, and optimization—impact the controllability-realism trade-off?
• RQ5 Can the model generalize to composite rules involving multiple objectives (e.g., stop at stop signs and avoid off-road)?

Key findings

• CTG achieved the lowest rule violation (0.0132) and high realism (0.456) in the ablation study under the speed limit rule, outperforming variants without dynamics enforcement or guidance.
• With 5 inner optimization steps, CTG reduced rule violation to 0.0156, though realism deviation slightly increased, indicating a trade-off between control and fidelity.
• Guidance combined with filtration reduced rule violations by 90% compared to no guidance, demonstrating the effectiveness of filtering guided samples.
• The variant that only supervised actions (without state supervision) had lower realism deviation (0.2057) but higher failure rates (0.2824), showing the importance of state-level regularization.
• Enforcing unicycle dynamics was critical: removing it increased rule violation to 1.2797 and realism deviation to 3.4204, severely degrading performance.
• On the nuScenes dataset, CTG achieved top-two performance in both rule violation and realism deviation across all evaluated rules, including composite rules like stop signs and no off-road.