Aug 2026· Applied Sciences· 0 citations· 13 references
TL;DR
A Large Language Model-enhanced Autonomous Reinforcement Learning Penetration Testing framework that leverages the domain knowledge embedded in a Large Language Model to perform tactical planning, thereby pruning the original action space into a compact set of candidate actions.
Abstract
In recent years, Deep Reinforcement Learning (DRL) has emerged as a promising approach for automating penetration testing due to its capability to perform sequential decision-making in complex environments. However, in real-world enterprise networks, attack actions are typically characterized by highly coupled multi-dimensional parameter combinations, resulting in an exponentially expanding discrete action space. Such a large action space significantly degrades exploration efficiency and prevents conventional DRL agents from learning effective attack paths under sparse-reward conditions. To address these challenges, this paper proposes a Large Language Model-enhanced Autonomous Reinforcement Learning Penetration Testing framework (L-ARLPT). Specifically, the framework leverages the domain knowledge embedded in a Large Language Model (LLM) to perform tactical planning, thereby pruning the original action space into a compact set of candidate actions. Subsequently, an experience-driven layer employs the optimization mechanism of a Deep Q-Network (DQN) to conduct value estimation and policy learning within the reduced candidate set. To validate the effectiveness of the proposed framework, a high-fidelity enterprise penetration-testing simulation environment was constructed based on realistic enterprise attack scenarios. Experimental results demonstrate that, in a high-fidelity enterprise penetration-testing environment with a raw theoretical parameter-combination space containing 6×107 combinations, the proposed L-ARLPT framework achieves an average penetration depth of 3.33 out of 4.00, substantially outperforming both reinforcement learning baselines (all ≤1.27) and LLM-based baselines (1.12). Moreover, successful episodes require only 128.43 decision steps on average, enabling long-horizon, cross-domain lateral penetration in high-dimensional discrete action spaces.
In the quantitative finance area, particularly in order execution, reinforcement learning (RL) has shown great promise due to its ability to interact with market environments based on real data. However, traditional RL methods suffer from slow research speed and rely on static market assumptions, which do not consider the impact of the agent's execution action on the environment. To address these, we propose a Self-Evolutional single-agent/multi-agent Reinforcement Learning (SE-RL) framework. The framework utilizes a Large Language Model (LLM) to design various RL algorithm modules, such as agent model design, reward function, profiling, communication, and state imagination, by leveraging the LLM generating module output or code. SE-RL could continuously improve the accuracy of LLM-generated RL algorithms through a dual-enhancement kit at both high-level (prompt refinement) and low-level (parameter fine-tuning). Additionally, we use a multi-agent system to simulate dynamic financial markets, accounting for the impact of order executions on market dynamics. To further enhance training in such a dynamic market, we develop a hybrid environment training method that could rebalance each environment's loss weight. Comprehensive experiments on 200 realistic stock datasets demonstrate that our proposed framework outperforms current state-of-the-art baselines. Project page: https://kdd2026-se-rl.github.io/.
Vincent Fu, Xinxin Xu, Weichen Xu et al.· Proceedings of the 32nd ACM...· 0 citations
Despite recent advances in applying artificial intelligence (AI) techniques to radio access network (RAN), critical challenges remain: traditional machine learning (ML) algorithms suffer from limited generalization across varying network topologies, whereas general-purpose large language models (LLMs) face high computational demands and lack domain-specific knowledge. To address these gaps, this article introduces the evolving RAN intelligent controller (RIC) (EvoRIC) framework, a hierarchical architecture that enables continuous evolution by leveraging a non-real-time RIC (non-RT RIC) for global model updates and a near-real-time RIC (near-RT RIC) for local execution, dynamically empowering LLMs with domain-specific decision-making capabilities. Within this framework, we employ a reinforcement learning-based fine-tuning (RLFT) mechanism where an LLM operates as an actor within a proximal policy optimization (PPO) agent. By leveraging the interaction tuples collected from the wireless environment, the LLM's parameters are iteratively updated to align semantic reasoning with rigorous network performance objectives. We evaluate the generalization and efficacy of the proposed EvoRIC framework within integrated access and backhaul (IAB) networks, and finally, discuss the open challenges and future directions of the EvoRIC framework toward realizing autonomous O-RAN.
Lingyan Bao, Jemin Lee, Tony Q. S. Quek· 0 citations
Delta (Differential Testing for DRL Agents) is proposed, a novel and comprehensive framework that automatically identifies both safety-critical and optimality bugs in DRL agents and investigates the effectiveness of three offline RL algorithms in generating challenger agents.
Junda He, Jieke Shi, Zhou Yang et al.· 0 citations
A novel HRL model is proposed that supports direct off-policy correction based on a Flow-based Deep Generative Model (FDGM) that leverages the inverse operation of FDGM to achieve goals aligned with the current knowledge of the lower-level policy.
Jaeyoon Kim, Junyu Xuan, C. Liang et al.· Journal of Artificial Intell...· 0 citations
The integration of Large Language Models (LLMs) with Reinforcement Learning (RL) for financial decision-making has grown rapidly in recent years, yet the literature remains fragmented and lacks systematic comparison across methods. In this survey we analyze 34 core studies (2023–2026), selected through a multi-stage process involving 84 initial candidates and 46 full-text reviews, and propose a three-paradigm taxonomy (feature-based, auxiliary-based, and policy-based) based on the functional role of LLMs within the RL pipeline. Analysis of these integration paradigms reveals an emergent architectural trade-off: while tighter policy-based coupling theoretically offers deeper contextual reasoning, it frequently introduces significant computational overhead and training instability. Conversely, simpler feature-based integration provides superior scalability and stability, though often at the expense of representational depth. Given the current benchmark fragmentation, the reported performance gains across these studies remain difficult to validate universally across different asset classes. Critical gaps identified include the insufficient handling of data leakage and look-ahead bias, standardized benchmarks, and limited alignment with regulatory frameworks such as MiFID II and the EU AI Act.
Ghusoon Hadi al-Aldaffaie, Alireza Taheri, Amirfarhad Farhadi et al.· Discover Artificial Intellig...· 0 citations
This work proposes TAPO: Transition-Aware Policy Optimization for LLM Agents, a unified training framework that alternates between policy optimization and transition supervision, and demonstrates that TAPO consistently improves task performance over pure policy optimization baselines.
Cong Li, Peixi Peng, Yisen Zhao et al.· 0 citations