Skip to content

End-to-end Conditional Diffusion for Realistic and Controllable Visual Traffic Scenario Generation

Jul 2026 · arXiv.org · Vol abs/2607.18637 · 0 citations · 50 references
Computer Science

TL;DR

Conditioned on front-view visual observations, E2E-CDiff jointly denoises future motion states and executable low-level controls for route-interacting background vehicles, and mitigates the planning-control mismatch in conventional two-stage trajectory-then-controller pipelines.

Abstract

Generating closed-loop traffic scenarios that are both realistic and controllable is crucial for evaluating autonomous driving systems, especially under rare safety-critical interactions. Existing learning-based methods often struggle to balance controllability and realism, offering either limited fine-grained control over traffic behavior or controllable scenarios at the expense of behavioral plausibility. This paper presents E2E-CDiff, an end-to-end conditional diffusion framework for controllable and realistic scenario generation. Conditioned on front-view visual observations, E2E-CDiff jointly denoises future motion states and executable low-level controls for route-interacting background vehicles. This unified state-action generation mitigates the planning-control mismatch in conventional two-stage trajectory-then-controller pipelines. Differentiable guidance further regulates speed, enforces drivable-area compliance, and supports collision-avoidance or collision-seeking behaviors, enabling both naturalistic and safety-critical scenario generation. Experiments on Bench2Drive show that E2E-CDiff achieves a favorable controllability-realism trade-off compared with representative reinforcement- and imitation-learning baselines, while its collision-guided variant induces challenging interactions across multiple autonomous driving systems. E2E-CDiff also performs competitively as a learning-based ego planner, demonstrating the generality of end-to-end state-action diffusion.

View source

Similar papers

Preprint Oct 2026

Controllable and Photorealistic Pedestrian Risky Motion Generation for End-to-End Driving Safety Evaluation

Evaluating end-to-end autonomous driving under rare, safety-critical vehicle-pedestrian interactions requires photorealistic, sensor-level scenarios. However, trajectory-based scenario generators cannot synthesize raw visual observations, whereas video-based approaches lack controllability. To bridge this gap, we prese...

Si-Yuan Liu, Miao Li, Hai-Bao Yu et al. · 0 citations
Preprint Sep 2026

Safety-Critical Scenarios Emerge from Initial Scenes

Safety-critical driving scenario generation has largely focused on manipulating the behavior of surrounding agents while starting from an initial scene from driving data. This assumption can limit the space of discoverable failures, since driving data can provide little opportunity for meaningful interaction. For examp...

Yin Wu, Jia-Rong Wei, Carl Esselborn et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Collision Snapshot Guided Time-Reversed Safety-Critical Scenario Generation

The generation of safety-critical traffic scenarios is essential for training and evaluating autonomous vehicles. Prior approaches typically perturb the trajectories of existing agents in a traffic scenario using simplified adversarial objectives to induce safety-critical interactions, which can limit the plausibility...

Taehyung Kim, Jongeun Choi · 0 citations
Sep 2026

ZYT-World: A Real-Time Controllable World Model for Closed-Loop Autonomous-Driving Simulation

This work presents ZYT-World, a single architecture that natively generates four fisheye views with field of view>180{\deg} and three pinhole views and a 19M-parameter variational autoencoder decoder decoder, W8A8 quantization, and the inference engine reduce decoding, backbone, and incremental-execution costs, respect...

Bo-Ni Hu, Xiong Wei, Hao-Ming Huang et al. · 1 citation · ⚡1
Preprint Aug 2026

Knowledge-Data-Dual-Driven Reinforcement Learning for Autonomous Vehicle Control in Mixed Traffic

This work proposes Knowledge-Data Dual-driven Reinforcement Learning (KDDRL), a conditional deep generative model that effectively handles intention uncertainty, accelerates training convergence, and outperforms conventional baseline methods in terms of safety, efficiency, and comfort.

Jie Fang, Wei Zheng, Mengyun Xu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.