When an off-the-shelf coding agent is used directly as a robot policy, observing a browser-based 3D interface through screenshots and acting by posing a virtual target gripper through a few tools, the agent's harness, its prompts, tools, and control rules, largely determines success, and until now it has been written b...
Zhi-Jie Wei, Ferris Tan, Jing-Hui Wang et al.· 2 citations
Unreliable visual inputs can harm task performance and cause potential physical safety risks for vision-language-action (VLA) models. We analyze how $\pi 0.5$ and GR00T models act under input faults such as image blackouts and freezing. We find that blackout and freezing produce distinct physical failure modes even whe...
Heejae Suh, Jongwook Han, Zahra Gholami et al.· 0 citations
Frontier models can control robots, but reasoning through every reach, grasp, and retreat makes manipulation slow and token-intensive. We revisit code as policy with a different division of labor: models build executable tools, code handles multi-phase motions, and models decide what to do next. We introduce URAI (Univ...
S. Ge, Alex Zhou, Jian-Shu Zeng et al.· 2 citations
Pilot, an efficient engine that reduces sparse inference overhead through lightweight scoring and cross-step reuse of token selections, is introduced, an efficient engine that reduces sparse inference overhead through lightweight scoring and cross-step reuse of token selections.
Xin-Ling Xie, Hao-Dong Wang, Jia-Zhi Mi et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Coding agents powered by large language models (LLMs) have shown remarkable abilities to autonomously reason about and achieve goals in the digital world. However, bringing this success to the physical world remains challenging. On the one hand, direct generation methods (e.g., Code as Policies) often suffer from the L...
Jiaheng Hu, Roberto Martín-Martín, Peter Stone et al.· 4 citations· ⚡1
This work presents DrivingBench, to its knowledge the first benchmark where general-purpose vision-language models must drive a real car, and details the design principles behind the action interface, and shows how the tool output format and the framing of the task combined to determine whether models would drive at al...
Aditya Ramabadran, Simon Mahns, Tobias Gessler· 0 citations
This work designs a hybrid architecture that combines sparse anchors with an offset refinement module for efficient multi-modal trajectory prediction and adopts a two-stage training paradigm consisting of pretraining and reward-guided fine-tuning, demonstrating superior performance under real-time constraints.
Cheng-Lin Chen, Lu-Jia Wang, Xin-Hu Zheng et al.· 0 citations
Vision-Language-Action (VLA) models based on generative frameworks, such as Flow Matching, have recently achieved impressive performance in robotic manipulation. Unlike deterministic policies, Flow Matching enables VLA models to learn conditional action trajectory distributions, where latent noise vectors induce differ...
Gong-Xin Yao, Yong-Shen Zhao, Jia-Yi Deng et al.· 0 citations
TERRA provides a practical route from scene-less motion data to muscle-actuated locomotion over diverse non-flat terrain and improves terrain accuracy, sharply reduces anatomical and interaction violations, and achieves the highest observed completion rate over supported terrain families.
Merkourios Simos, Chengkun Li, Bianca Ziliotto et al.· 0 citations
GestAdapt is presented, a workspace-conditioned framework that conditions co-speech gesture generation on a prescribed wrist workspace and learns from six complementary co-speech corpora through a shared motion representation and supports retargeting to different robot embodiments.
Bo-Song Ding, Xiang-Ling Zhang, Miao Xin et al.· 0 citations
Synergy-based Imitation Learning is introduced, a novel framework for automated, label-free demonstration quality assessment in offline reinforcement learning and achieves performance comparable to, and in sparse-reward human teleoperation scenarios, superior to, offline reinforcement learning trained on true environme...
Yuto Tanaka, Kyo Kutsuzawa, Martina Doku et al.· 0 citations
Multi-Agent Path Finding (MAPF) aims to find collision-free paths for multiple agents in a shared environment. Classical MAPF assumes that all static obstacles are known in advance, but real-world environments can change unexpectedly due to fallen objects, spills, or other local disturbances. When such changes are spat...
Viraj Parimi, Shao-Hung Chan, Han Zhang et al.· 0 citations
A new system identifies and corrects unreliable object descriptions stored in a robot’s memory, improving the robot’s ability to interact with its environment.
Robots are getting smarter, but how can their hardware match that growth? New Microsoft Research findings show that moving AI inference beyond the robot can improve task success, boost efficiency, and support more advanced physical AI workloads. The post Offloaded inference for real-world physical AI robotics appeared first on Microsoft Research.
