Skip to content

Acting in Meters: Learning Metric Interactions for Precise Robotic Manipulation

Sep 2026 · arXiv.org · Vol abs/2609.18243 · 1 citation · 48 references
Computer Science

TL;DR

A metric interaction framework that models object-level and scene-level interactions in physical Cartesian space at a shared metric scale and improves diverse VLA and WAM baselines with a small number of additional parameters and training steps is introduced.

Abstract

Vision-Language-Action models and World-Action Models have advanced language-conditioned robotic manipulation, yet often leave metric relations among actions, objects, and scene geometry implicit. Human manipulation combines semantic understanding of task-relevant objects with spatial feedback that guides hand motion relative to objects and their surroundings. Inspired by this, we introduce a metric interaction framework that models object-level and scene-level interactions in physical Cartesian space at a shared metric scale. At the object level, Interaction-Centric Tokens (ICTs) explicitly represent end-effector pose trajectories relative to manipulated objects and are jointly denoised with actions, providing physically grounded interaction supervision. At the scene level, the Metric Action Interaction Field (MAIF) uses action and ICT queries to attend to metric scene point-cloud features and learns geometry-conditioned action corrections. Through two-stage adaptation, our framework improves diverse VLA and WAM baselines with a small number of additional parameters and training steps. Experiments demonstrate average success-rate gains of 0.80 and 3.59 percentage points on LIBERO and RoboTwin 2.0, respectively, alongside gains of 6.80 percentage points on real-world tasks and 7.45 percentage points on their out-of-distribution variants.

View source

Similar papers

Preprint Oct 2026

UNITAS: A 3D-Native World Action Model for Embodied Manipulation

World action models (WAMs) aim to answer a coupled physical question: given a task instruction, what motion should the robot execute, and how will that motion change the surrounding world? Most existing WAMs build on pretrained video generators and represent world evolution through images or visual latents. Robotic int...

Rui-Xiang Wang, Yong-Yi Su, Wen-Lve Zhou et al. · 0 citations
Preprint Oct 2026

UniWAM: Unified World-Action Model

UniWAM is introduced, a unified architecture that integrates a physical reasoner, a world generator, and an action predictor to jointly learn semantic understanding of the physical world, visual generation, and action prediction and uncovers a log-linear scaling law of unified human-robot co-training.

Wen-Xuan Song, Jia-Yi Chen, Jing-Bo Wang et al. · 6 citations · ⚡2
Open access Aug 2026

SIRModel: Learning Spatial Intermediate Representation to Parameter-Efficiently Fine-Tune a Vision Language Model for Manipulation

A spatial Gaussian-guided hierarchical framework that uses ordered 3D Gaussian interaction regions as an explicit planning interface between vision–language reasoning and action generation and demonstrates the effectiveness of explicit 3D interaction representations for spatial reasoning and long-horizon manipulation.

Li Lin, Ming-Hao Shi, Teng-Long Wang · 0 citations
Preprint Oct 2026

World Action Learning via Interaction-Centric Spectral Latent Guidance

Results show that interaction-centric spectral guidance provides an effective and scalable way to transfer physical interaction knowledge from human egocentric video to robot control.

Zhi-Ming Liu, Yi-Kun Miao, Ying Chen et al. · 1 citation
Sep 2026

MachEmbodied-U0: Unified Understanding and Generation Model for Embodied Intelligence

MachEmbodied-U0, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture, is presented, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture that combines competitive...

Hao-Ran Wen, Wen-Fu Wang, Kun-Song Shi et al. · 1 citation
#artificial intelligence Preprint Oct 2026

EgoLAP: Learning from Egocentric Human Data through Language-Action Reasoning

Egocentric human data offer a path to scaling robot learning beyond costly robot demonstrations, yet the embodiment gap makes raw human trajectories a poor supervisory target for control. Our key insight is that, although low-level actions are embodiment-specific, their underlying motion intent can capture task-relevan...

Li-Han Zha, Shresth Grover, Tenny Yin et al. · 0 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.