Reinforcement-Learning-Driven Improved MOEA/D Algorithm for Configuration Optimization of Four-Way Shuttle Warehouse Systems

Abstract

The four-way shuttle warehouse system (FWSWS), as a form of high-density automated warehousing equipment, is characterized by high storage density, high space utilization, and operational flexibility. The initial configuration of this warehousing system is constrained by spatial layout and equipment resources, exhibiting discrete, strongly coupled, and nonlinear characteristics. When solving such complex constrained problems, existing algorithms like MOEA/D and NSGA-II generally suffer from insufficient constraint-handling capabilities and low search efficiency for feasible solutions. To address these issues, this paper proposes the RL-IMOEA/D algorithm, integrating reinforcement learning and a knowledge-driven strategy (RL-IMOEA/D). This algorithm enhances the capability of searching for feasible solutions through a knowledge-driven feasible region repair mechanism and utilizes Q-learning to dynamically adjust the variable neighborhood search (VNS) strategy, thereby improving the convergence and diversity of the Pareto solution set. Experimental results show that, compared with other optimization algorithms, RL-IMOEA/D achieves an average hypervolume (HV) value of 1.2068—a 0.96% improvement over the standard MOEA/D baseline (1.1953)—while its inverted generational distance (IGD) value is reduced from 0.1487 to 0.0411. The proposed algorithm demonstrates superior performance in terms of convergence, solution diversity, and constraint-handling capability. The obtained configuration schemes satisfy the capacity and queueing stability constraints, verifying the effectiveness of the proposed algorithm in complex discrete optimization problems and providing an effective solution approach for the initial configuration optimization of four-way shuttle warehouse systems.