Towards Automated Reinforcement Learning: Applications and Prospects of Large Language Models as Cognitive Components in the Full RL Lifecycle

Abstract

: Deep reinforcement learning (DRL) has made significant progress in recent years. However, it still faces multiple bottlenecks in real-world applications. Agents suffer from low sample efficiency, and designing effective reward functions remains very difficult. Furthermore, traditional DRL lacks general cognitive abilities. Recent advances in large language models (LLMs) provide a promising solution. By utilizing the strong coding and reasoning skills of LLMs, researchers can overcome many of these limitations. Addressing the bottlenecks of low sample efficiency and difficult reward design in real-world deep reinforcement learning applications, this paper constructs a taxonomic framework of "large language model-driven reinforcement learning throughout its entire lifecycle," systematically exploring the proxy mechanism of large models for human experts in four stages: environment, reward, decision-making, and collaboration. This study finds that the code generation capabilities of large models enable the automated construction of simulated scenarios and dense rewards;the "hierarchical planning" effectively balances the common-sense nature and real-time nature of decision-making, while the natural language interface significantly enhances the interpretability of multi-agent collaboration. This indicates that the deep integration of large models and reinforcement learning achieves a closed loop of cognition and action, providing a key technical path for building artificial general intelligence.