Evolving Maze Environments for Deep Q-Learning via Evolutionary Algorithms

Abstract

The performance and generalization of deep reinforcement learning (RL) agents are strongly influenced by the structure of their training environments, yet environment design is typically manual and rarely optimized in a principled manner. Among them, maze-solving tasks present several challenges for evaluating reinforcement learning methods in navigation and sequential decision-making. In this paper, we propose a Maze Evolution framework, denoted as ME, where an evolutionary algorithm (EA) searches a constrained space of grid-based mazes to produce a best evolved environment set for RL with desirable properties (e.g., feasibility and controlled structural difficulty). The resulting evolved environment set is then used to train a Deep Q-Network (DQN) agent. The performance of ME is evaluated on a test environment set and compared against RL agents that is trained on a fixed environment. Experimental results demonstrates that our method can increase the success rate of RL agent by 12% to achieve its goal.