Safe Decision-Making via Adaptive Causal Representation for Autonomous Driving.
TL;DR
This work proposes a safe offline-to-online decision-making framework with adaptive causal representation, an adaptive causal transformer (AC-Transformer), which learns a causal representation from offline driving trajectories that is adaptive as the deployment distribution evolves.
Abstract
Offline reinforcement learning (RL) is promising for autonomous driving, but as deployment conditions drift away from the offline training distribution, policies may encounter out-of-distribution (OOD) scenarios, such as unseen road geometries and diverse driving behaviors, rendering offline-learned decisions unreliable. To address this issue, we propose a safe offline-to-online decision-making framework with adaptive causal representation. At its core is an adaptive causal transformer (AC-Transformer), which learns a causal representation from offline driving trajectories. The representation is causal because it captures how states and actions influence long-horizon reward and safety outcomes under deployment shift. As these influences evolve over sequential interactions, we model them jointly over states, actions, rewards, and costs. Then, bisimulation regularization is introduced to further organize the learned representation into a consequence-consistent latent structure, so that it reflects long-horizon consequences rather than superficial traffic patterns. During online deployment, a phase-adaptive objective (PAC) is designed to progressively refine the learned representation, making it adaptive as the deployment distribution evolves. We evaluate the proposed method in five distinct driving scenarios against eight representative baselines. The results demonstrate stronger generalization across diverse road structures and stochastic driving styles while preserving a favorable balance between safety and efficiency.