This work introduces a unified RL formulation that jointly optimizes agent and environment policies, where the environment policy learns graph edge costs to provide global movement guidance via backward Dijkstra search and achieves significant improvements over the strong search-based planner, Causal-PIBT, across multiple high-density maps.
Abstract
Lifelong Multi-Agent Path Finding (LMAPF) requires repeatedly planning collision-free paths for agents that continuously receive new goals upon reaching their current ones. While many learning-based planners have been proposed for LMAPF, most rely on oversimplified kinematic assumptions that may overlook motion constraints critical to real-world performance. In this work, we study a more realistic LMAPF model derived from many real-world automated warehouse systems, termed LMAPF-R2, which incorporates robust safety constraints and in-place rotation constraints. These constraints substantially increase coordination difficulty, particularly in highly constrained spaces. To address these challenges, we propose Search-Aided Joint Reinforcement Learning (SJRL). We first augment neural policies with Causal PIBT, a single-step search-based planner that resolves agents'collisions and propagates their intentions. We then introduce a unified RL formulation that jointly optimizes agent and environment policies, where the environment policy learns graph edge costs to provide global movement guidance via backward Dijkstra search. Experiments demonstrate that SJRL achieves significant improvements over the strong search-based planner, Causal-PIBT, across multiple high-density maps. We further validate SJRL in a challenging mixed-reality warehouse environment with 8 physical robots and 248 virtual robots.
This work presents a prioritized Safe Interval Path Planning algorithm (SIPP-PP) with a novel limited goal reservation strategy to prevent goal-blocking conflicts while allowing shared goal regions, and demonstrates a multi-robot planner capable of real-time operation in dense scenarios, satisfying the stringent requirements of industrial applications such as drive units in fulfillment centers.
Rajat Kumar, Kristin Predeck, Ken Meszaros et al.· 0 citations
: This paper provides a thorough survey and integrative presentation of cooperative path planning for multi-robot systems operating in dynamic, cluttered, and partially observable environments. People synthesise algorithmic foundations ranging from heuristic graph search to sampling-based motion planners, including A*, D* Lite, and Safe Interval Path Planning for discrete/time-augmented spaces, as well as RRT, RRT*, and Informed RRT* for continuous configuration spaces. Multi-agent coordination techniques are reviewed, covering reciprocal collision avoidance (ORCA) and centralised Multi-Agent Path Finding (MAPF) solvers such as Conflict-Based Search (CBS) and bounded-suboptimal variants (ECBS). The paper also examine control and safety layers like Model Predictive Control and Control Barrier Functions that translate plans into dynamically feasible commands with safety guarantees. Recent progress in cooperative multi-agent reinforcement learning (MAPPO, QMIX, MADDPG) is evaluated for adaptability under partial observability and nonstationary environments. Applications in warehousing, intelligent transportation, and disaster response are used to illustrate practical trade-offs and integration patterns, referencing real-world systems such as Kiva-style warehouse fleets and autonomous driving pipelines. The paper concludes with a focused discussion on open challenges — scalability with guarantees, safety under uncertainty, sim-to-real transfer, and planning – control interface fragility — and proposes research directions including learning-augmented heuristics, unified safety-aware planning, adaptive MPC – CBF filters, and more informative benchmarks to drive reproducible progress.
Yun Pan· Proceedings of the 3rd Inter...· 0 citations
Coordinated multi-agent exploration requires not only efficient individual coverage but also non-redundant coverage across agents over extended planning horizons. Conventional approaches rely on hand-crafted coordination rules, while end-to-end multi-agent learning methods are difficult to scale and train. Diffusion-based planners such as DARE offer a promising alternative by generating long-horizon trajectories instead of single-step actions, but existing methods are trained on a narrow planner distribution, limiting behavioral diversity and inference-time controllability. We propose a Planner-Conditioned Diffusion Policy (PCDP) for graph-based multi-agent exploration. PCDP is trained on demonstrations from multiple planner styles with planner identity as an explicit conditioning input, enabling a single shared model to learn a multimodal trajectory distribution and generate diverse, controllable trajectory candidates from the same observation. Rather than learning coordination end-to-end, we reuse this multimodal single-agent policy across all agents and introduce coordination through local reranking, in which nearby agents jointly select the trajectory combination with minimal predicted overlap. We evaluate PCDP against classical and diffusion-based baselines on 100 held-out maps in a four-agent simulation setting. PCDP matches the perfect success rate of the diffusion-based baselines while improving mean max-agent travel, total team travel, and agent imbalance. Crucially, reranking alone over a single-planner baseline yields only marginal gains, indicating that planner-conditioned multimodality is the main contributor to improved coordination. Qualitative simulation results and real-robot experiments with two agents further validate that diverse long-horizon trajectory generation produces emergent spatial separation between agents without any explicit repulsion mechanism.
Marcus Yu Siong Teo, Jeric Lew, T. Duhan et al.· 0 citations
Multi-agent path planning (MAPP) under partial observability requires agents to coordinate their movements and complete tasks efficiently without access to global information. The planning space and coordination complexity grow rapidly with increasing numbers of agents, targets, and obstacles. We formulate large-scale MAPP as a partially observable networked Markov decision process. Based on this formulation, we propose a decentralized model-based Actor-Critic using the Kronecker-factored trust region (DM-ACKTR) algorithm. The algorithm integrates local model learning with ACKTR-based policy optimization in an independent learning architecture. Each agent learns a local model to predict the next observation and reward. These predictions are used to construct additional transitions for Actor and Critic updates. A neighborhood-based communication mechanism incorporates information from nearby agents into value estimation. Region partitioning reduces each agent’s effective planning space. These improvements enable DM-ACKTR to continue outperforming the baseline algorithms as the scale of the MAPP problem increases. Experiments across three training and five evaluation scenarios show that DM-ACKTR achieves the best overall performance. Among the five evaluated algorithms, it consistently obtains the highest TCR and lowest CR, improving TCR by 2.06–4.35% and reducing CR by 11.26–25.95% relative to the respective best baselines.
Ye-Min Liu, Jinhao Yang, Xiang-Yu Ma et al.· Electronics· 0 citations