Skip to content

MAAP: Multi-Agent Active Perception for Collaborative Manipulation

Sep 2026 · arXiv.org · Vol abs/2609.21929 · 1 citation · 39 references
Computer Science

TL;DR

MAAP (Multi-Agent Active Perception), in which every arm is dual-purpose: it executes manipulation actions and, through the wrist camera it carries, simultaneously serves as a moving viewpoint for the team, can serve as an active perception mechanism in its own right.

Abstract

Multi-agent manipulation naturally produces multiple task-driven viewpoints: every arm carries a wrist camera and moves through the scene while acting. Yet these observations are typically underutilized, and active perception in manipulation is still often treated as requiring a dedicated sensing agent. We introduce MAAP (Multi-Agent Active Perception), in which every arm is dual-purpose: it executes manipulation actions and, through the wrist camera it carries, simultaneously serves as a moving viewpoint for the team. We pair this with RAIL (Role-Aware Imitation Learning), a controller that predicts each arm's current role alongside its action chunk and conditions action generation on it, representing role-dependent actions within one network. Across four simulated tasks, widening the perception regime lifts average success from 56.5% with a fixed camera to 62.5% with one active wrist view and 70.0% with all of them, while MAAP+RAIL reaches 79.2%. RAIL's additional gain is concentrated on the three-arm Microwave task, where success rises from 47% to 82% on identical multi-wrist inputs. On a dual-arm platform, MAAP+RAIL succeeds in 14 of 20 placement trials compared with 0 of 20 for fixed-view ACT. Collaborative manipulation can thus serve as an active perception mechanism in its own right.

View source

Similar papers

Preprint Oct 2026

LiveAct: Towards Real-Time Multi-Agent World Models with Heterogeneous Action Interfaces

Interactive video world models predict how an environment evolves under user actions, but most existing systems assume one controlled agent and one task-specific action interface. We study multi-agent video world modeling, where one model generates a joint visual future for multiple controlled agents while interpreting...

Xinyu Wang, Hua-Feng Shi, Yue Ma et al. · 0 citations
Preprint Sep 2026

MoPA: Coordinated Mobile Manipulation via Subsystem-Specific Perception Alignment

MoPA is presented, a framework that aligns perceptual conditioning with mobility and manipulation while preserving coordination at the action level, and achieves state-of-the-art performance across all three task suites.

Guang-Yu Chen, Qi-Wei Liang, Shao-Long Zhu et al. · 3 citations
Sep 2026

From Passive Execution to Active Exploration: Agentic Embodied Manipulation in Realistic Environments

This work proposes an agent-based active exploration framework that enables robots to dynamically interact with the environment rather than merely execute predefined instructions, and introduces a fine-grained perception-execution interleaving strategy, which tightly couples visual feedback with skill execution to impr...

Shi-Lin Ma, Chu-Bin Zhang, Xu-Long Bai et al. · 0 citations
Sep 2026

LIBERO-Agent: Evaluating General-Purpose Agents for Direct Embodied Manipulation

General-purpose agents can plan, use tools, and revise their behavior from feedback, but it remains unclear whether these capabilities transfer from digital environments to embodied manipulation. To investigate this question, we introduce LIBERO-Agent, an agent-native benchmark for evaluating these agents in robot mani...

Zi-Jie Diao, Yi-Tong Chen, Si-Cheng Xie et al. · 5 citations
Preprint Sep 2026

MM-ABC: Towards Generalist Mobile Manipulation via Seeing, Coordinating and Imagining

This work pretrain MM-ABC, a foundation model built around Seeing, Coordinating, and Imagining Arm-Base Collaboration, which combines sparse multi-level VLM features for spatial perception; a training-only future branch that uses world imagination and geometric intent as extra supervision, strengthening perception and...

Qi-Wei Liang, Guang-Yu Chen, Shao-Long Zhu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.