Aug 2026· IEEE Access· Vol 14, pp. 149288-149310· 0 citations· 41 references
Computer ScienceEngineering
TL;DR
This study presents a centralized safety aware M2M framework for cooperative goal-directed navigation in a heterogeneous mobile robot system composed of a vision-capable robot and a cameraless robotic vehicle that can approve safe motion, trigger conservative replanning or holding behavior, and preserve a strict separation between semantic reasoning and executable control.
Abstract
Safe coordination in heterogeneous machine-to-machine (M2M) robotic systems is difficult when robots differ in sensing capability, environmental awareness, and motion execution roles. This study presents a centralized safety aware M2M framework for cooperative goal-directed navigation in a heterogeneous mobile robot system composed of a vision-capable robot and a cameraless robotic vehicle. The objective is to guide the robots toward a detected goal region, such as a traversable target area or open door direction, while avoiding static and dynamic obstacles and preventing unsafe inter robot interactions. The central cooperation principle is shared perception: the vision-capable robot provides semantic environmental awareness through a centralized server, allowing the camera-less robot to act using this shared scene representation together with its own odometry, IMU, and state feedback. Both robots communicate with the server through an MQTT broker and continuously publish robot-state data, while the vision capable robot additionally transmits visual observations. A vision language model interprets the scene, and the extracted semantic information is converted into conservative geometric constraints, including obstacle regions, traversable areas, goal regions, safe corridors, and motion boundaries. A large language model supports high level task allocation reasoning by proposing robot specific navigation decisions, while deterministic controllers remain responsible for low-level execution. Before any command is issued, each candidate action is verified by a zonotope based reachability engine that checks obstacle avoidance, safe corridor containment, and inter-robot collision constraints. Only commands satisfying these reachability-based safety conditions are approved and transmitted to the corresponding robot. Online validation in clear-path and dynamic-obstacle scenarios demonstrates that the framework can approve safe motion, trigger conservative replanning or holding behavior, and preserve a strict separation between semantic reasoning and executable control. The proposed framework unifies shared semantic perception, broker-based M2M communication, cooperative task allocation, and formal reachability verification to support safer coordination of heterogeneous mobile robots with asymmetric sensing capabilities.
This work introduces DuoMind, a distributed hierarchical framework for multi-robot coordination through semantic communication, and develops RoboPoly, a benchmark comprising long-horizon manipulation tasks that require coordinated, closed-loop execution under distributed control.
Han-Chu Zhou, De-Chen Gao, Hang Wang et al.· 1 citation
This work introduces RobotAgentBench (RAB), a comprehensive benchmark for evaluating robot agents in realistic, open-world household environments and enables systematic comparisons of standalone VLA/WAM models and diverse robot agent architectures under a unified planning-execution-verification framework.
Ling-Jun Mao, Jing-Lin Cao, Lu-Kun He et al.· 1 citation
Foundation models are increasingly used for high-level multi-robot task planning, yet independently generated symbolic plans can become jointly infeasible after geometric grounding because of shared spatial and temporal constraints. We study execution-aware coordination of large-language-model (LLM) planners for multi-...
V. Khatana, Zi-Jian Song, N. Hovakimyan et al.· 1 citation
RoboTalk is introduced, a synthetic data-generation pipeline and dataset of 7,950 multimodal trajectories spanning 53 mobile-manipulation kitchen tasks for training small VLMs to communicate and coordinate and fine-tuning open-source models on this dataset can reach 77% success on novel held-out tasks.
Dorian Benhamou Goldfajn, Mason Nakamura, Saaduddin Mahmud et al.· arXiv.org· 0 citations
We present CoPlanVLM, which uses a Vision Language Model (VLM) to translate a single natural-language instruction and image into coordinated paths for a team of two robots. While VLMs offer a flexible, natural-language interface for commanding robot teams, their weak spatial reasoning has greatly limited VLM-based path...
David R. Martin, Jared Paquet, M. A. Al Faruque et al.· 0 citations
The visual system is a central component of mobile robot navigation, as it provides essential information about the surrounding environment for motion planning and coordination. However, navigation requires the robot to interpret sensory data through localization and mapping processes, such as Simultaneous Localization...
Abdirakhman Onabek, Zaki Al-Farabi, Ilyas Umurbekov et al.· Scientific Journal of Astana...· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduJul 30, 2026
From feet to fingertips — we are teaching robots intelligent whole-body control, fine dexterity, and teamwork to complete a broad range of complex tasks.