Skip to content

Planning in the Backbone: DiffAdapterVLA for Native Continuous Trajectory Generation with Driving VLMs

Sep 2026 · 0 citations · 61 references
Computer Science

TL;DR

DiffAdapterVLA is introduced, which realizes Planning in the Backbone: it injects explicit trajectory tokens into selected VLM late layers, bringing trajectory state into backbone forward computation, where it co-evolves with driving conditions at different depths.

Abstract

Pretrained driving vision-language models (VLMs) integrate visual, route, language, and driving context into rich driving priors, yet their representation objectives remain separated from continuous driving planning. Existing methods typically begin trajectory generation only after the VLM has formed a final condition, leaving depth-wise condition computation outside the stepwise formation of trajectory state. We introduce DiffAdapterVLA, which realizes Planning in the Backbone: it injects explicit trajectory tokens into selected VLM late layers, bringing trajectory state into backbone forward computation, where it co-evolves with driving conditions at different depths. Lightweight layer-wise DiffAdapters organize this computation into recursive trajectory refinement, while asymmetric joint attention preserves directed guidance from the condition stream to trajectory planning. By placing planning within existing backbone computation rather than relying on an independent trajectory planner, DiffAdapterVLA adapts only lightweight trajectory modules to turn existing driving priors into efficient continuous planning capability. NAVSIM results show that it achieves high-quality closed-loop planning with low end-to-end latency using few trainable parameters, and demonstrate that jointly evolving trajectory state and depth-wise driving conditions in VLM late-layer computation effectively realizes continuous trajectory planning.

View source

Similar papers

Preprint Aug 2026

MomADv2: Reliable Temporal Memory for End-to-End Autonomous Driving

MomADv2, a reliable state-space memory framework for long-horizon end-to-end autonomous driving, introduces a Selective State-Space Planning Memory Query Module, which filters historical planning queries based on temporal continuity and command consistency, and models the evolution of planning intentions through a sele...

Zi-Ying Song, Sheng-Kai Zhang, Lin Liu et al. · 0 citations
Sep 2026

FeasibleFlow: One-Step Joint Transport of Configuration Feasibility and Trajectories for End-to-End Driving

This work proposes FeasibleFlow, a one-step end-to-end generative framework that jointly transports a configuration-space feasibility field and multimodal ego trajectories and introduces the Anchor-relative ranker (ARR) and Pareto-ReinFlow to balance safety and progress in candidate selection and generation.

Xiang Li, Bi-Kun Wang, Qing Xu et al. · 0 citations
Preprint Aug 2026

VTInstructor: Visual Trajectory Prompting for Navigation Instruction Generation in Continuous Environments

Navigation instruction generation from ego-centric RGB video in continuous environments is an important yet challenging task for human-robot interaction and scalable dataset construction. Prior instruction generators assume discrete viewpoint graphs with panoramic observations, where trajectory structure is explicit; i...

Hao-Lin Yang, Yu-Xing Long, Zi-Han Yang et al. · 0 citations
#artificial intelligence Preprint Aug 2026

Aligning Multi-Trajectory Supervision with Policy Optimization for VLA Driving

A novel framework that aligns multi-trajectory supervision with policy optimization, and introduces two complementary mechanisms: feasibility-first advantage assignment and dynamic distillation to ensure that expanded trajectory supervision is effectively absorbed during policy optimization.

Tian Zhang, Zhuo Huang, Hong-Rui Ye et al. · 0 citations
#artificial intelligence Preprint Sep 2026

DiffWAM: A Fast and Efficient Navigation World Action Model

DiffWAM is presented, a geometry-conditioned navigation world-action model that directly transforms multi-level predictive features into continuous camera trajectories, providing a direct alternative to generate-then-reconstruct navigation pipelines.

Morui Zhu, Yu-Ze Wu, Xi-Jie Huang et al. · 1 citation
Preprint Oct 2026

StageVLN: Spatial and Trajectory Auxiliary Guidance for Efficient Vision-Language Navigation

Vision-and-Language Navigation (VLN) policies increasingly benefit from strong semantic priors provided by large vision-language models (VLMs). However, standard action supervision does not explicitly encourage intermediate representations to preserve scene geometry, relative orientation, or global episode progress. In...

Anh Dao, Q. Phạm, Le Danh Vinh et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.