Skip to content

Safe Multi-Robot Coordination via VLM–LLM Reasoning and Reachability Analysis

Aug 2026 · IEEE Access · Vol 14, pp. 149288-149310 · 0 citations · 41 references
Computer Science Engineering

TL;DR

This study presents a centralized safety aware M2M framework for cooperative goal-directed navigation in a heterogeneous mobile robot system composed of a vision-capable robot and a cameraless robotic vehicle that can approve safe motion, trigger conservative replanning or holding behavior, and preserve a strict separation between semantic reasoning and executable control.

Abstract

Safe coordination in heterogeneous machine-to-machine (M2M) robotic systems is difficult when robots differ in sensing capability, environmental awareness, and motion execution roles. This study presents a centralized safety aware M2M framework for cooperative goal-directed navigation in a heterogeneous mobile robot system composed of a vision-capable robot and a cameraless robotic vehicle. The objective is to guide the robots toward a detected goal region, such as a traversable target area or open door direction, while avoiding static and dynamic obstacles and preventing unsafe inter robot interactions. The central cooperation principle is shared perception: the vision-capable robot provides semantic environmental awareness through a centralized server, allowing the camera-less robot to act using this shared scene representation together with its own odometry, IMU, and state feedback. Both robots communicate with the server through an MQTT broker and continuously publish robot-state data, while the vision capable robot additionally transmits visual observations. A vision language model interprets the scene, and the extracted semantic information is converted into conservative geometric constraints, including obstacle regions, traversable areas, goal regions, safe corridors, and motion boundaries. A large language model supports high level task allocation reasoning by proposing robot specific navigation decisions, while deterministic controllers remain responsible for low-level execution. Before any command is issued, each candidate action is verified by a zonotope based reachability engine that checks obstacle avoidance, safe corridor containment, and inter-robot collision constraints. Only commands satisfying these reachability-based safety conditions are approved and transmitted to the corresponding robot. Online validation in clear-path and dynamic-obstacle scenarios demonstrates that the framework can approve safe motion, trigger conservative replanning or holding behavior, and preserve a strict separation between semantic reasoning and executable control. The proposed framework unifies shared semantic perception, broker-based M2M communication, cooperative task allocation, and formal reachability verification to support safer coordination of heterogeneous mobile robots with asymmetric sensing capabilities.

Read PDF

Similar papers

#artificial intelligence Preprint Oct 2026

DuoMind: Enabling Distributed Multi-Robot Coordination with Semantic Communication

This work introduces DuoMind, a distributed hierarchical framework for multi-robot coordination through semantic communication, and develops RoboPoly, a benchmark comprising long-horizon manipulation tasks that require coordinated, closed-loop execution under distributed control.

Han-Chu Zhou, De-Chen Gao, Hang Wang et al. · 1 citation
Preprint Oct 2026

RobotAgentBench: Towards Open-world Robot Autonomy

This work introduces RobotAgentBench (RAB), a comprehensive benchmark for evaluating robot agents in realistic, open-world household environments and enables systematic comparisons of standalone VLA/WAM models and diverse robot agent architectures under a unified planning-execution-verification framework.

Ling-Jun Mao, Jing-Lin Cao, Lu-Kun He et al. · 1 citation
Preprint Oct 2026

Verifier-Mediated Multi-LLM Coordination for Multi-Robot Path Planning

Foundation models are increasingly used for high-level multi-robot task planning, yet independently generated symbolic plans can become jointly infeasible after geometric grounding because of shared spatial and temporal constraints. We study execution-aware coordination of large-language-model (LLM) planners for multi-...

V. Khatana, Zi-Jian Song, N. Hovakimyan et al. · 1 citation
#robotics Sep 2026

RoboTalk: Learning Multi-Robot Communication and Coordination from Multimodal Demonstrations

RoboTalk is introduced, a synthetic data-generation pipeline and dataset of 7,950 multimodal trajectories spanning 53 mobile-manipulation kitchen tasks for training small VLMs to communicate and coordinate and fine-tuning open-source models on this dataset can reach 77% success on novel held-out tasks.

Dorian Benhamou Goldfajn, Mason Nakamura, Saaduddin Mahmud et al. · 0 citations
Preprint Oct 2026

CoPlanVLM: Coordinated Path Planning for a Robot Team using Vision Language Models

We present CoPlanVLM, which uses a Vision Language Model (VLM) to translate a single natural-language instruction and image into coordinated paths for a team of two robots. While VLMs offer a flexible, natural-language interface for commanding robot teams, their weak spatial reasoning has greatly limited VLM-based path...

David R. Martin, Jared Paquet, M. A. Al Faruque et al. · 0 citations
Open access Sep 2026

EFFICIENT MULTI-ROBOT SLAM IN A LABYRINTH ENVIRONMENT: A CENTRALIZED APPROACH USING ALGEBRAIC CONNECTIVITY AUGMENTATION

The visual system is a central component of mobile robot navigation, as it provides essential information about the surrounding environment for motion planning and coordination. However, navigation requires the robot to interpret sensory data through localization and mapping processes, such as Simultaneous Localization...

Abdirakhman Onabek, Zaki Al-Farabi, Ilyas Umurbekov et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.