Achieving robust humanoid locomotion in complex, unstructured environments requires transitioning from reactive proprioception to proactive perception. However, integrating exteroception remains a significant challenge: existing approaches often fail to achieve high-speed, stable locomotion across diverse, unstructured...
Shaoting Zhu, Ziwen Zhuang, Mengjie Zhao et al.· 0 citations
Generative AI (GenAI) is rapidly advancing the field of Autonomous Driving (AD), extending beyond traditional applications in text, image, and video generation. We explore how generative models can enhance automotive tasks, such as static map creation, dynamic scenario generation, trajectory forecasting, and vehicle mo...
Katharina Winter, Abhishek Vivekanandan, Rupert Polley et al.· 0 citations
We present NeuSE, a novel Neural SE(3)-Equivariant Embedding for objects, and illustrate how it supports object-based Simultaneous Localization and Mapping (SLAM) for consistent spatial understanding with long-term scene changes. NeuSE is a set of latent object embeddings created from partial object observations. It se...
Jiahui Fu, Yilun Du, Kurran Singh et al.· 0 citations
Agentic robotics places a large language model (LLM) agent in the robot control loop, where it composes perception, motion, and vision-language-action (VLA) tools under the guidance of a memory and thereby generalizes to new tasks without retraining. This generality comes at a steep price in efficiency. Every primitive...
Bing-Xin Xu, Wen-Rui Bao, Xin-Xin Liu et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
How should demonstration quality be used during vision-language-action (VLA) pretraining to improve performance after target-task fine-tuning? We compare weighting, filtering, and conditioning across reward definitions and label granularities using 5,000 naturally suboptimal human demonstrations and 24 target tasks. Wi...
Ming-Xuan Yan, Li-Tian Gong, Di Huang et al.· 0 citations
Robot policies are expected to generalize across deployment environments, including variations in the robot itself: cameras are bumped and remounted, calibrations drift, controllers run at different rates, and end-effector pads wear down. We present LIBERO-RIG, a benchmark that perturbs only the robot's hardware and se...
Andrea Yaoyun Cui, Han-Yu Wang, Xin-Yi Wang et al.· 1 citation
It is difficult to accurately model the physics of the interaction between a robot and the objects it manipulates, including contact mechanics, coupled contact dynamics, deformations, stiffness and friction. We present RoboSimStudio (RSS), an interactive toolkit that makes choices of solver, contact model, and physical...
Kaiyuan Chen, Shuangyu Xie, Uksang Yoo et al.· 0 citations
In cooperative Multi-Agent Reinforcement Learning (MARL), agents must learn to coordinate their actions to achieve shared objectives. A key indicator of effective coordination is behavioral diversity, i.e., the agents'potential to enact distinct yet complementary control policies given the same set of observations. Alt...
Frontier language-model agents can now improve robots on their own: they write control code, test it, and keep what works as skills and memories for the next task. We introduce SEAR (Self-Evolving Agentic Robotics), a framework and benchmark built around two questions. Is the agent really learning, or just trying more...
Xin-Yi Wang, Bang-Zheng Li, Han-Yu Wang et al.· 3 citations
Steertuning is introduced, a system that enables robots to learn new tasks in new environments from 5 human demonstrations in under 5 minutes, and achieves a 71% success rate across 12 tasks in 12 new environments, ranging from lab settings to kitchens in new homes.
Billy Yan, Isaac Tu, Kelly Lee et al.· 0 citations
World models (WMs) and world-action models (WAMs) focus on future world prediction and executable action generation, respectively. Although these models learn shared physical dynamics and interaction semantics from embodied trajectories, they are typically trained in isolation, which restricts knowledge reuse between p...
Jingbo Xu, Jianjie Fang, Zhuohang Li et al.· 0 citations
3D semantic occupancy prediction is crucial for autonomous driving, but camera-based methods degrade in low-light conditions and LiDAR-based methods are sensitive to adverse weather. 4D radar is more robust, yet its sparsity and the high cost of radar tensor processing limit its use. We propose Radar Geometric-Semantic...
Christofel Rio Goenawan, Dong-Hee Paek, Seung-Hyun Kong· 0 citations
A new system identifies and corrects unreliable object descriptions stored in a robot’s memory, improving the robot’s ability to interact with its environment.
Robots are getting smarter, but how can their hardware match that growth? New Microsoft Research findings show that moving AI inference beyond the robot can improve task success, boost efficiency, and support more advanced physical AI workloads. The post Offloaded inference for real-world physical AI robotics appeared first on Microsoft Research.