H-VLA is proposed, a hierarchical VLA framework that decouples high-level key-action reasoning from low-level motion generation and adopts a two-stage training strategy that emphasizes key-action reasoning during pre-training and dense motion generation during fine-tuning.
Abstract
Vision-Language-Action (VLA) models have shown strong potential for robotic manipulation, but many existing methods still rely on direct mappings from language and visual observations to dense actions. This formulation can weaken the semantic reasoning capability inherited from pre-trained Vision-Language Models (VLMs), which are mainly optimized for visual-linguistic understanding rather than low-level control, and becomes fragile under spatial variations, including changes in object positions, scene layouts, robot embodiments, and camera viewpoints. To address these limitations, we propose H-VLA, a hierarchical VLA framework that decouples high-level key-action reasoning from low-level motion generation. H-VLA combines a Key-Action Model for predicting a key-action as the next manipulation subgoal, a Motion Planning Model for generating dense future actions conditioned on the predicted key-action, and a Unified Camera-Centric Action Space for consistent representation across datasets, embodiments, and viewpoints. We further adopt a two-stage training strategy that emphasizes key-action reasoning during pre-training and dense motion generation during fine-tuning. Experiments show that H-VLA achieves strong performance on SimplerEnv, reaching 91% on Google Robot visual matching, 84% on Google Robot variant aggregation, and 81% on WidowX visual matching. On Agilex real-robot tasks, H-VLA improves over the strongest baseline by 10, 47, and 16 percentage points under in-distribution, out-of-distribution position, and out-of-distribution scene/object settings, respectively.
Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet adapting generalist vision-language-action (VLA) policies to real-world humanoid loco-manipulation remains challenging. Unlike fixed-base or wheeled manipulators, humanoids must coordinate locomotion, posture, gaze, and dua...
Lang-Zhe Gu, Cheng-Kai Hou, Meng Li et al.· 0 citations
UniWAM is introduced, a unified architecture that integrates a physical reasoner, a world generator, and an action predictor to jointly learn semantic understanding of the physical world, visual generation, and action prediction and uncovers a log-linear scaling law of unified human-robot co-training.
Wen-Xuan Song, Jia-Yi Chen, Jing-Bo Wang et al.· 6 citations· ⚡2
A symmetric Dual-Arm Expert (DAE) architecture built upon a shared Vision-Language Model (VLM) backbone with decoupled, arm-specific expert towers is proposed, providing preliminary evidence of emergent skill generalization from single- to dual-arm tasks (as well as the reverse), together with cross-arm motion-domain s...
Yong-Shen Zhao, Han Gao, Bao-Ping Cheng et al.· arXiv.org· 0 citations
PG-VLA (Perception-Guided VLA), a planner that keeps task-specific perception and motion prediction outside the VLM while using their outputs as structured planning context, achieves strong safety-oriented performance without reinforcement fine-tuning, explicit reasoning supervision, or dedicated bird’s-eye-view (BEV)...
Л. М. Иванов, D. Yudin· Optical Memory and Neural Ne...· 0 citations
Vision-language-action (VLA) models provide a unified framework for visual understanding, language grounding, and robot control, yet anticipating how the scene will evolve beyond the current observation can provide additional information for control. Action supervision alone does not explicitly encourage such anticipat...
Zi-Le Zhong, Yu-Xin Du, Tao Lin et al.· 0 citations
PDRVLA is introduced, whose lightweight Physical Dynamics Module extracts physical tokens from vision--language features to condition action generation without online video generation, andlations show improved future prediction with world-model supervision and evaluate how subtask conditioning, future object-pose super...
Han-Qing Wang, Zhao-Tian Cai, Ruichuan An et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.