Aug 2026· Frontiers in Robotics and AI· Vol 13· 0 citations· 38 references
Medicine
TL;DR
CoMuRoS enables runtime, event-driven replanning on physical robots and supports flexible multi-robot and human-robot collaboration across diverse scenarios.
Abstract
Heterogeneous multi-robot teams require systems that can interpret natural-language goals, allocate tasks, and adapt to unexpected events. We developed CoMuRoS (Collaborative Multi-Robot System), a generalizable hierarchical architecture combining a centralized task-manager LLM with decentralized robot-level LLMs for executable Python generation from primitive ROS2 skills. The task manager uses static planning rules and dynamic context, including task history, robot/task status, and detected events, while onboard perception using VLM/image processing classifies events as relevant or irrelevant and triggers replanning. Hardware experiments demonstrated recovery from disruptive events, filtering of irrelevant distractions, and coordinated transport with emergent human-robot cooperation, achieving success rates of 9/10 for collaborative object recovery, 8/8 for coordinated transport, and 5/5 for human-assisted recovery. Simulation studies demonstrated intention-aware replanning. A curated benchmark of 22 scenarios, 54 tasks, and around 20 robots evaluated task allocation, classification, IoU, executability, and correctness across multiple LLMs, with correctness up to 0.91 ± 0.053; a 20-scenario replanning benchmark achieved Correctness = 0.948 ± 0.034 using Grok 3. CoMuRoS enables runtime, event-driven replanning on physical robots and supports flexible multi-robot and human-robot collaboration across diverse scenarios.
Multi-robot systems provide the parallelism and redundancy necessary for long-horizon tasks, while Large Language Models (LLMs) offer the reasoning capabilities to decompose these objectives into actionable plans. However, effectively grounding this high-level reasoning in physical multi-robot execution remains an open challenge. Existing LLM-based approaches fall mainly into two categories: Single-robot methods achieve robust contact-rich manipulation but lack the coordination mechanisms required for tasks spanning multiple workspaces. Current multi-robot frameworks focus on high-level planning, often treating manipulation as an idealized primitive that fails to account for real-world execution uncertainties. To address this, we propose a hierarchical closed-loop agentic LLM-based framework to ensure robust multi-robot manipulation. Our system consists of three specialized agents: the Planning Agent decomposes instructions into allocated sub-tasks, the Manipulation Agent for each robot executes actions via adaptive tool use, and the Verification Agent closes the loop by monitoring physical outcomes and feeding back semantic corrections. Extensive real-world experiments demonstrate that our framework achieves superior success rates, ensures robust adaptability ranging from single to cross workspace manipulation, and offers a generalizable approach for diverse manipulation tasks.
Yi-Xiang He, Lan Wei, Haoming Cen et al.· Robotics· 0 citations
Multi-robot systems, particularly heterogeneous robot swarms, can improve the efficiency of complex task execution through parallel collaboration and complementary capabilities. However, conventional rule-based methods rely on predefined task models and specialized decision making programs, making it difficult to understand complex semantic instructions and coordinate heterogeneous robots. LLMs introduce strong language understanding and task reasoning capabilities, allowing multi-robot systems to interpret instructions, decompose tasks, and assign roles according to task semantics. VLMs further incorporate visual perception, enabling robots to reason about objects, regions, and spatial relationships in physical environments. Nevertheless, existing LLM/VLM based methods often depend on known maps, centralized and synchronized decision making, limiting their generalization to heterogeneous robots and unseen tasks. We therefore propose a framework that combines decentralized asynchronous reasoning, lightweight information sharing, capability aware collaboration, and a unified action interface, enabling general purpose VLMs to generate robot specific actions executed by learning free experts without task or robot specific training. Experiments across diverse scenarios and multiple VLMs show success rates above 70\%, with completion time reduced by up to 55.8\% relative to the geometric greedy baseline.
Yuan Zhou, Ruitong Lin, Shen Wang et al.· 0 citations
Robotic industrial panel operation requires not only accurate control localization but also compliance with operating procedures, safety rules, and device-state constraints distributed across heterogeneous manuals. This study presents MaCoPlanner, a task-planning framework built on knowledge compiled from equipment manuals that converts equipment manuals into a typed intermediate representation, retrieves task- and state-relevant evidence, and uses it to support plan generation. Before actuation, candidate plans are symbolically rolled out and checked against procedural and state-transition constraints; detected violations are localized and returned for targeted repair, while unresolved plans are rejected. A separate execution interface grounds verified symbolic actions to physical controls and updates the device state. Under an independent evaluation oracle, MaCoPlanner achieves a final violation rate of 2.7%, and 26.3% of the runs in the repair analysis are rejected after exhausting the refinement budget. Compared with Raw-Manual, task success increases from 62.8% to 84.4% on Level-2 tasks and from 25.9% to 43.2% on Level-3 tasks. Experiments on a controller-panel simulator without an attached industrial load further demonstrate integrated execution feasibility under representative interaction conditions, without claiming industrial deployment readiness.
Gui-Peng Xin, Jiahe Xua, Mohammad Deghat et al.· 0 citations
Long-horizon robotic tasks require diverse capabilities that no single policy can reliably provide. Heterogeneous policies offer complementary strengths, but orchestrating them requires reasoning over uncertain capability boundaries and cross-policy distribution mismatch, which are largely overlooked by existing planning methods built on homogeneous, predefined skills with fixed applicability. We propose RoboHarness, a unified framework that encapsulates independently developed robot control systems as reusable agentic skills. Although instantiated in this work with VLAs, RL policies, and task-and-motion planning (TAMP) systems, RoboHarness is designed as a general framework compatible with a broader range of robot policies, such as navigation policies, model predictive controllers, and world-action models. RoboHarness uses multi-modal execution memory and online evidence to characterize policy capability boundaries for capability-aware decomposition and routing. To stabilize policy handoffs, its Memory Bridge retrieves execution trajectories associated with the next policy, estimates its in-distribution state region, and guides the robot toward that region without joint policy retraining. Extensive experiments on three public benchmarks, 500 customized tasks, and 135 real-robot experiments demonstrate effective capability-aware routing and stable policy orchestration, yielding substantial improvements in zero-shot long-horizon planning and out-of-distribution robustness.
Jinbang Huang, Yuan Hu, Zhiyuan Li et al.· 1 citation· ⚡1
Humanoid robots are emerging as flexible robotic resources for autonomous manufacturing systems, where different types of tasks must be assigned to suitable robots while shared production resources are coordinated effectively. However, realistic manufacturing environments involve dynamic task arrivals, event-driven priority changes, heterogeneous robot capabilities, and shared-resource contention. In addition, digital twin-based scheduling may rely on state information that is delayed or uncertain, which can reduce the reliability of scheduling decisions. To address this issue, this paper extends the previously proposed deep reinforcement learning-based concurrency control (DRLCC) framework for robustness-aware scheduling and shared-resource control of a multi-functional humanoid robot team. The extended framework integrates capability-aware task assignment, feasibility-based action masking, and priority ceiling protocol (PCP)-based shared-resource coordination under delayed and uncertain digital twin observations. The framework is evaluated in a humanoid-based autonomous manufacturing scenario using performance indicators including task completion, urgent-task delay, resource contention, humanoid utilization, and robustness degradation under imperfect state feedback. Compared with the greedy ceiling-based baseline, DRLCC reduces high-priority task delay by approximately 10.9%, priority inversions by 42.1%, average waiting time by 73.8%, average block count by 74.0%, and temporary infeasible events by 73.5%, while maintaining comparable humanoid utilization. The robustness analysis further shows that overall task-completion performance remains stable under imperfect digital twin feedback, although coordination-level metrics are more sensitive to observation uncertainty and delay. These results suggest that the extended DRLCC framework can support robustness-aware humanoid robot team scheduling in autonomous manufacturing environments where digital twin observations are delayed or uncertain.
R. Anwar, Won-Tae Kim· Applied Sciences· 0 citations
