Empirical evaluations reveal a fundamental brittleness in existing defenses: with a single trainable 7B planner, Trident reduces blue agent defensive performance by an average of 522% compared to static red agent baselines while autonomously discovering emergent behaviors such as decoy avoidance and adaptive state prioritization that static heuristics entirely fail to uncover.
Abstract
Autonomous cyber defense systems based on Deep Reinforcement Learning (DRL) have attracted significant research attention, yet remain evaluated almost exclusively against static, heuristic red agents, leaving their robustness against adaptive threats critically understudied. Meanwhile, recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have improved LLM reasoning, but their integration into cybersecurity remains elusive due to the absence of suitable benchmark environments and interaction datasets. To bridge this gap, we introduce Trident, an agentic LLM red teaming framework comprising three components: a dynamic benchmark with isolated sandbox servers spanning CybORG CAGE 4 and CyberWheel, a dataset comprises over 13,000 high-fidelity red-blue interaction trajectories for RLVR, and a ``Code-as-Policy''RLVR agentic architecture Trident Agentic). The latter reformulates red agent training as a contextual bandit via a tripartite Log Summarizer--Planner--Coder design, where a trainable Planner generates complete attack strategies from compressed execution logs, which a frozen Coder translates into executable Python policies deployed against live DRL defenders. Empirical evaluations reveal a fundamental brittleness in existing defenses: with a single trainable 7B planner, Trident reduces blue agent defensive performance by an average of 522% compared to static red agent baselines while autonomously discovering emergent behaviors such as decoy avoidance and adaptive state prioritization that static heuristics entirely fail to uncover.
The use of a Large Language Model (LLM) to improve autonomous defensive decision-making within an ACO environment is investigated and an online policy distillation framework is proposed that transfers the LLM's defensive policy into a lightweight RL agent containing only 64,910 parameters, reducing model size by several orders of magnitude while maintaining effective defensive capabilities.
Konur Tholl, F. Rivest, Mariam El Mezouar et al.· 0 citations
As cyber threats continue to evolve, there is a need for Autonomous Cyber Defense (ACD) strategies capable of fast and context-aware responses. Reinforcement learning (RL) has shown promise in automating cyber defense by exploring and learning effective countermeasures. However, RL often struggles with sparse reward signals and insufficient context to handle diverse attack scenarios. Furthermore, the convergence time of an RL agent is often high, making it difficult to train the agent in online settings. To address these challenges, we propose a large language model (LLM)-enhanced RL method that builds and queries a knowledge base (KB) derived from agent–environment interactions. We leverage the pre-trained knowledge of an LLM on different cybersecurity frameworks and use the LLM to analyze parts of the KB to generate appropriate actions for the RL agent. The LLM-generated output is infused into the RL training process to improve performance and reduce convergence time. To validate our approach, we formulate two RL problems: a contextual bandit problem, which accounts for possible misclassifications of network flows by the detection module, and a multi-step RL problem, which considers that adversarial actions may be missed by monitoring or detection tools. For the contextual bandit problem, we develop a custom environment guided by the MITRE ATT&CK framework, while for the multi-step RL problem, we use a prominent Cybersecurity simulation platform named CybORG. Experimental results show that our proposed approach outperforms the baseline RL by over 75% and 65% in the contextual bandit and multi-step RL settings, respectively, in terms of selecting more effective actions.
Md. Shamim Towhid, Shahrear Iqbal, E. P. Neto et al.· IEEE Transactions on Network...· 0 citations
A systematic analysis of 207 studies selected from 4447 records following the PRISMA 2020 guidelines, covering work published between 2020 and 2026 across cybersecurity and computer vision finds systems that are robust against adaptive adversaries, interpretable under operational constraints, and auditable in environments where AI accountability is a legal requirement.
Reinforcement learning for coding agents increasingly relies on long-running agent harnesses to manage tool integration, repository contexts, and execution feedback. However, the native execution environments of these harnesses are inherently misaligned with policy-gradient training: environmental crashes and reward hacking corrupt outcome signals, while train-inference discrepancies decouple rollout behavior from policy updates. To address this, we present LEGO-RL, a framework that bridges native coding-agent harnesses with scalable policy-gradient optimization without modifying their internal control flow. LEGO-RL is built upon three pillars: (1) faithful optimization via in-process LLM proxying that captures raw generation streams for token-level alignment and robust trainer-side log-probability recomputation, even under harness-side compaction or re-serialization; (2) reliable execution via scalable sandbox orchestration featuring image caching and stage-wise defenses to mitigate reward hacking; and (3) observable training through an integrated plugin that automates validation and monitoring, paired with a Live UI for granular trajectory diagnostics. We evaluate LEGO-RL by training the sparse MoE model Qwen3.5-35B-A3B with GSPO across three native coding-agent harnesses. LEGO-RL improves Qwen3.5-35B-A3B across OpenHands SDK (64.0% to 70.4%), Claude Code (62.4% to 68.2%), and OpenCode (57.2% to 66.6%) on SWE-bench Verified, while maintaining a rollout-training probability correlation above 0.99.
Yiming Du, Yuxin Jiang, Tao Yuan et al.· 0 citations
As cyber attacks grow more sophisticated, defenders need autonomous systems that are fast, adaptable, and explainable. Over the last decade, various strategies have been proposed for automated cyber defence (as opposed to static rule-based or signature-based), including those based on reinforcement learning (RL). Researchers have proposed various algorithms to improve RL-based defenders and evaluated them using simulation-based frameworks like the DARPA CAGE-2. Although RL showed promise, it has many limitations, for example, the lack of a realworld training environment and the need for extensive training, which is time-consuming. In this case study, we investigate whether LLM agents can be used instead of RL agents to automate cyber defence. Large Language Models (LLMs) can reason over natural language and generalize from extensive pretraining. They are attractive for cyber defence because they can read textbased system states and make human-like, explainable decisions. We propose a unique way to convert CAGE-2 states to natural language and a domain-specific fine-tuning method that improve the average reward and reduce hallucination significantly, beating existing RL-based agents and state-of-the-art LLM agents.
Arijit Diganto, S. Lohrasbi, Euclides Carlos Pinto Neto et al.· International Conference on...· 0 citations
A Large Language Model-enhanced Autonomous Reinforcement Learning Penetration Testing framework that leverages the domain knowledge embedded in a Large Language Model to perform tactical planning, thereby pruning the original action space into a compact set of candidate actions.