This work proposes Language-Structured Relational Q-Learning, instantiated through an Ego-Centric Relational Q-Network (ERQ-Net), which jointly learns inter-vehicle relevance and action values from dynamic traffic graphs, and formalises this discrepancy as a recognition-control gap.
Abstract
Natural-language-based scenario generation offers an intuitive means of describing rare and complex driving interactions, yet it is still uncertain whether training with language-structured data leads to truly adaptive control policies. We propose Language-Structured Relational Q-Learning, instantiated through an Ego-Centric Relational Q-Network (ERQ-Net), which jointly learns inter-vehicle relevance and action values from dynamic traffic graphs. Language descriptions define surrounding-vehicle behaviours during training, while prompts and semantic actor roles are hidden from the policy. ERQ-Net must therefore infer threat relevance solely from observable kinematics and interactions. Across 2,500 safety-critical scenarios, language-structured training improves test success from 49-52% to 55-58% and increases adversary-focused attention from 1.2x to 2.1x, demonstrating emergent threat awareness. However, this representational gain does not consistently translate into adaptive control: trained policies perform similarly to the best constant action, while a portfolio of simple policies solves 76% of scenarios. We formalise this discrepancy as a recognition-control gap and show that reward reweighting and margin shaping do not eliminate the resulting policy collapse. Evaluations of realism, criticality, semantic accuracy, and transfer of state-interface representations to CARLA further highlight both the strengths and the constraints of language-structured relational policy learning in safety-critical driving scenarios.
Offline reinforcement learning (RL) is promising for autonomous driving, but as deployment conditions drift away from the offline training distribution, policies may encounter out-of-distribution (OOD) scenarios, such as unseen road geometries and diverse driving behaviors, rendering offline-learned decisions unreliable. To address this issue, we propose a safe offline-to-online decision-making framework with adaptive causal representation. At its core is an adaptive causal transformer (AC-Transformer), which learns a causal representation from offline driving trajectories. The representation is causal because it captures how states and actions influence long-horizon reward and safety outcomes under deployment shift. As these influences evolve over sequential interactions, we model them jointly over states, actions, rewards, and costs. Then, bisimulation regularization is introduced to further organize the learned representation into a consequence-consistent latent structure, so that it reflects long-horizon consequences rather than superficial traffic patterns. During online deployment, a phase-adaptive objective (PAC) is designed to progressively refine the learned representation, making it adaptive as the deployment distribution evolves. We evaluate the proposed method in five distinct driving scenarios against eight representative baselines. The results demonstrate stronger generalization across diverse road structures and stochastic driving styles while preserving a favorable balance between safety and efficiency.
Chenyang Zhao, Haoen Huang, Depeng Li et al.· IEEE Transactions on Neural...· 0 citations
Vision-language-action policies rely on large multimodal backbones to jointly perform perception, language conditioning, and action generation at every control step. Much of this capacity supports open-domain semantics, whereas continuous robot manipulation primarily requires compact representations of observations, actions, and the transitions induced by actions. Pixel-level world models provide another route, but predicting visual details irrelevant to control can be unnecessarily expensive. We propose SLIM (Self-supervised Latent Interaction Model), a compact 0.5B-parameter latent interaction policy. SLIM learns action-grounded predictive latents that capture both action-conditioned future transitions and the actions that explain observed changes. SLIM learns these representations through self-supervised masked trajectory prediction, combining action reconstruction with future-latent prediction. A compact Mixture-of-Transformers (MoT) backbone models interactions between observation latents and action tokens. The resulting policy is trained with flow matching for language-conditioned action generation. Across simulation benchmarks and real-world evaluation, SLIM matches or exceeds representative large-scale VLA and world-action-model baselines with fewer parameters, no additional embodied pretraining, lower inference latency, and substantially lower GPU memory usage.
Jingkai Wang, Zihan Tang, Gu Zhang et al.· 0 citations
We present lg-sfDA6-X, a language-grounded strategy-following distributed attentional actor architecture after conditional attention, for multi-agent deep reinforcement learning (MADRL). The proposed architecture aims to enable controllable and coordinated agent behaviors in application systems by leveraging a shared saliency representation that integrates environmental conditions with high-level textual instructions provided by external experts and users. To this end, we introduce language-based destination strategies, which allow agents to adapt their behaviors simply by specifying the target regions of the environment using natural language. We evaluate lgsfDA6-X in an object collection game and analyze how agents modify their cooperative and coordinated behaviors in response to diverse, previously unseen, and compositional instructions by users. Experimental results demonstrate that lg-sfDA6-X effectively grounds linguistic instructions into semantically structured latent representations, enabling robust strategy following and coordination. These findings suggest that lg-sfDA6-X provides a promising approach for achieving flexible and interpretable control of coordinated behaviors in MADRL through natural language instructions.
Yoshinari Motokawa, Toshiharu Sugawara· Annual International Compute...· 0 citations
Fire-VLA is introduced, a failure-informed self-evolution framework that converts unresolved failures into privileged supervision for the next policy in autonomous-driving vision-language-action models.