BeMapper: BicNet and evolutionary-based multi-agent path planning with effective reinforcement.

Abstract

Despite recent advances in multi-agent path finding, achieving robust coordination in dynamic and crowded warehouse environments remains a bottleneck due to training instability and inefficient credit assignment. To address these challenges, we propose BeMapper, a novel evolutionary-augmented reinforcement learning framework that integrates a multi-agent bidirectionally-coordinated network (BicNet) with a distributed actor-critic architecture. Technically, our core novelty lies in three aspects: (1) A bidirectional feature fusion mechanism that enables agents to perceive collective spatial states beyond local observations; (2) An evolutionary-driven critic selection strategy that iteratively propagates high-performing models to accelerate convergence; (3) A multi-metric scoring system that incorporates success rate variance to penalize unstable behaviors and resolve credit assignment ambiguity. Extensive experiments demonstrate the superiority of BeMapper: it achieves a 98.66% mean success rate, outperforming state-of-the-art baselines Mapper (95.51%) and BicNet (93.78%) by 3.15%and 4.88%, respectively. Crucially, BeMapper yields a significantly higher average reward of 18.81, representing a relative improvement of 1.65 over Mapper and a substantial leap over BicNet's near-zero performance (0.04). Furthermore, in more crowded scenarios, BeMapper reduces the average travel steps to 36, being 5-9 steps shorter than competing methods, effectively enhancing operational throughput while ensuring robustness for large-scale industrial automation.