Neuroevolutionary reinforcement learning : an agent-based solution for multi-echelon inventory control

Abstract

Managing replenishment across multi-echelon inventory systems is challenging due to stochastic demand and inter-echelon dependencies, forcing a trade-off between holding costs and stockout penalties. This research addresses this within a Linear Multi-Echelon Inventory System (LMEIS) and a Divergent Multi-Echelon Inventory System (DMEIS). In the DMEIS, complex allocation and inter-node competition create rugged fitness landscapes where Proximal Policy Optimization (PPO) and similar Policy-Tuning methods struggle with premature convergence. To overcome this, we propose a Policy-Building framework utilizing Neuroevolution of Augmenting Topologies (NEAT) to simultaneously co-evolve policy structure and weights. In the LMEIS, NEAT achieved a 25.02% total cost reduction over PPO and 37.28% over a Q-Learning baseline, converging significantly faster. In the complex DMEIS, standard NEAT exhibited evolutionary stagnation. Consequently, a novel variant, Mutation-Augmented NEAT (M-NEAT), was developed. By augmenting crossover and mutation rates exclusively for less dominant species, M-NEAT reduced dominant-species streak duration by 64.06%. This achieved a 13.61% cost reduction over PPO with only 5.73% computational overhead. M-NEAT consistently develops a risk-averse strategy eliminating stockout penalties, establishing it as a highly efficient alternative to Policy-Tuning for complex inventory control.