Aug 2026· Applied Informatics· Vol 7, pp. 334· 0 citations· 48 references
TL;DR
A spatial Gaussian-guided hierarchical framework that uses ordered 3D Gaussian interaction regions as an explicit planning interface between vision–language reasoning and action generation and demonstrates the effectiveness of explicit 3D interaction representations for spatial reasoning and long-horizon manipulation.
Abstract
Long-horizon robotic manipulation requires a policy to bridge task-level semantic reasoning with metric three-dimensional interaction geometry. Existing vision–language–action policies usually acquire geometry implicitly from visual tokens or introduce deterministic intermediate variables only in the image plane, which rely on expensive human annotations and training cost. This article presents a spatial Gaussian-guided hierarchical framework that uses ordered 3D Gaussian interaction regions as an explicit planning interface between vision–language reasoning and action generation. The proposed framework enables efficient adaptation of a pretrained vision–language model for robotic manipulation tasks. First, an automatic geometric enhancement pipeline converts raw robot demonstration videos into near-, mid-, and late-stage Gaussian supervision through foreground extraction, metric depth estimation, stable camera aggregation, end-effector localization, 3D lifting, and temporal grouping, without requiring manual 3D interaction annotation. The generated Gaussian representations provide structured spatial guidance, where their covariance characterizes interaction-region extent and variability rather than fully calibrated physical uncertainty. Second, a shared vision–language backbone predicts structured subtasks and Gaussian interaction regions, while a conditional diffusion executor generates future action chunks under these semantic and geometric conditions. A trajectory-to-Gaussian likelihood objective explicitly encourages consistency between generated motions and the predicted spatial interaction plan. Experiments on a mixed real-robot dataset derived from LHManip and RH20T show that our method improves trajectory tracking success from 55.7% to 70.8% over a same-backbone direct VLA baseline. Closed-loop simulation evaluation on LIBERO with 80% backbone parameter frozen achieves 85.3% average task success, demonstrating the effectiveness of explicit 3D interaction representations for spatial reasoning and long-horizon manipulation.
Reliable robotic manipulation requires both semantic understanding and precise control in three-dimensional space. However, vision-language-action (VLA) foundations largely inherit visual representations from 2D vision-language pre-training, while spatial supervision is often introduced only during task-specific adapta...
Fu-Hao Li, Wen-Xuan Song, Qi-Zhou Huang et al.· 0 citations
This work proposes a parameter-efficient approach to fine-tune a pretrained VLM for autonomous navigation using an Imperative Learning paradigm, and introduces a unified end-to-end navigation pipeline for natural-language-driven robotic control.
Sebastian Berger, Katharina Winter, Fabian B. Flohr· arXiv.org· 0 citations
Vision-and-Language Navigation (VLN) policies increasingly benefit from strong semantic priors provided by large vision-language models (VLMs). However, standard action supervision does not explicitly encourage intermediate representations to preserve scene geometry, relative orientation, or global episode progress. In...
Anh Dao, Q. Phạm, Le Danh Vinh et al.· 0 citations
MachEmbodied-U0, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture, is presented, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture that combines competitive...
Hao-Ran Wen, Wen-Fu Wang, Kun-Song Shi et al.· arXiv.org· 1 citation
This work proposes a representation-alignment method that enables the injection of 3D spatial awareness without compromising the model's original expressivity, and effectively incorporates 3D spatial representations into VLA reasoning, achieving robust performance and enhanced spatial awareness across diverse manipulat...
Chao-Ran Zhu, Hengyi Wang, Cheng-Lin Cui et al.· 0 citations
A metric interaction framework that models object-level and scene-level interactions in physical Cartesian space at a shared metric scale and improves diverse VLA and WAM baselines with a small number of additional parameters and training steps is introduced.
Li-Jie Wang, Zheng Lu, Yi-Ming Wang et al.· arXiv.org· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.