MAAP (Multi-Agent Active Perception), in which every arm is dual-purpose: it executes manipulation actions and, through the wrist camera it carries, simultaneously serves as a moving viewpoint for the team, can serve as an active perception mechanism in its own right.
Abstract
Multi-agent manipulation naturally produces multiple task-driven viewpoints: every arm carries a wrist camera and moves through the scene while acting. Yet these observations are typically underutilized, and active perception in manipulation is still often treated as requiring a dedicated sensing agent. We introduce MAAP (Multi-Agent Active Perception), in which every arm is dual-purpose: it executes manipulation actions and, through the wrist camera it carries, simultaneously serves as a moving viewpoint for the team. We pair this with RAIL (Role-Aware Imitation Learning), a controller that predicts each arm's current role alongside its action chunk and conditions action generation on it, representing role-dependent actions within one network. Across four simulated tasks, widening the perception regime lifts average success from 56.5% with a fixed camera to 62.5% with one active wrist view and 70.0% with all of them, while MAAP+RAIL reaches 79.2%. RAIL's additional gain is concentrated on the three-arm Microwave task, where success rises from 47% to 82% on identical multi-wrist inputs. On a dual-arm platform, MAAP+RAIL succeeds in 14 of 20 placement trials compared with 0 of 20 for fixed-view ACT. Collaborative manipulation can thus serve as an active perception mechanism in its own right.
Improved robustness to the evaluated occlusions and distribution shifts is supported, addressing part of the gap between structured manipulation and less controlled environments.
Jia-Wei Geng, Zhen-Yang Liu, Yu-Zheng Wu et al.· 0 citations
Interactive video world models predict how an environment evolves under user actions, but most existing systems assume one controlled agent and one task-specific action interface. We study multi-agent video world modeling, where one model generates a joint visual future for multiple controlled agents while interpreting...
Xinyu Wang, Hua-Feng Shi, Yue Ma et al.· 0 citations
MoPA is presented, a framework that aligns perceptual conditioning with mobility and manipulation while preserving coordination at the action level, and achieves state-of-the-art performance across all three task suites.
Guang-Yu Chen, Qi-Wei Liang, Shao-Long Zhu et al.· 3 citations
This work proposes an agent-based active exploration framework that enables robots to dynamically interact with the environment rather than merely execute predefined instructions, and introduces a fine-grained perception-execution interleaving strategy, which tightly couples visual feedback with skill execution to impr...
Shi-Lin Ma, Chu-Bin Zhang, Xu-Long Bai et al.· arXiv.org· 0 citations
General-purpose agents can plan, use tools, and revise their behavior from feedback, but it remains unclear whether these capabilities transfer from digital environments to embodied manipulation. To investigate this question, we introduce LIBERO-Agent, an agent-native benchmark for evaluating these agents in robot mani...
This work pretrain MM-ABC, a foundation model built around Seeing, Coordinating, and Imagining Arm-Base Collaboration, which combines sparse multi-level VLM features for spatial perception; a training-only future branch that uses world imagination and geometric intent as extra supervision, strengthening perception and...
Qi-Wei Liang, Guang-Yu Chen, Shao-Long Zhu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.