AuraForge is introduced to synthesize and validate executable security tests for training secure coding agents, and it is demonstrated that AuraForge provides more diverse and reliable security supervision to train secure coding agents.
Abstract
Coding agents are now proficient enough to generate complex software applications from a single prompt. As their capabilities have grown, human oversight has increasingly shifted from line-by-line code review toward hands-off evaluation of outcomes. However, recent studies have shown that such a transition exposes a critical risk: functional correctness alone does not guarantee a secure implementation. Despite growing attention to code security, training safer coding agents remains challenging because reliable security supervision is difficult to obtain at scale from real-world repositories. We introduce AuraForge to synthesize and validate executable security tests for training secure coding agents. Our approach combines attack-oriented test synthesis, language-extensible task construction, and safeguards against reward hacking. Using AuraForge, we construct AuraGym, a multi-language and multi-CWE executable training gym: 679 executable feature-implementation tasks from 344 real-world repositories across Python, JavaScript, and TypeScript, covering 177 CWE categories. On the subset with human-written security tests, AuraForge produces about 3 times as many test cases on average and reduces the false-positive rate by 83.23%, allowing alternative secure implementations to receive correct supervision. Training Qwen3.5-4B with synthesized security tests gains larger improvements than human-written security tests (average 19.7 FuncPass and 6.2 SecPass vs. 14.9 FuncPass and 4.4 SecPass) on three languages. These results demonstrate that AuraForge provides more diverse and reliable security supervision to train secure coding agents.
falcon{}, a counterexample-grounded verification framework that separates attack exploration from behavioral verification and connects them through execution traces of the program under test, is introduced, suggesting that execution-grounded feedback can mitigate reward overoptimization while preserving useful guidance...
Tue Le, A. N. Hoang, Gabriel Sasseville et al.· 0 citations
As vibe coding becomes increasingly capable and widespread, security vulnerabilities in even functionally correct solutions are a growing concern. When investigating functionally correct but insecure solutions, we find that the insecure agent is less than half as likely to conduct effective planning and testing for the...
Dan-Qing Wang, Bao-Lin Peng, Zhepei Wei et al.· 0 citations
MACGen is a multi-agent framework that integrates planning, security analysis, code synthesis and refinement to jointly optimize security and functionality and improves F&S@1 over direct prompting on CWEval and BaxBench.
Miseon Yu, Jaehoon Choi, Younghan Lee et al.· 0 citations
AI coding agents now write much of the code that ships, yet whether that code is secure remains unclear. We introduce the Last Secure Code Benchmark, a comprehensive evaluation of secure code generation by coding agents. Built from 150 real vulnerabilities, its 450 tasks span seven widely used languages (Java, C, C++,...
Tian-Yi Wu, Chao Ge, Peng Chen et al.· 0 citations
Large language models (LLMs) increasingly power agents that access sensitive information, use external tools, and modify software repositories. Although these capabilities offer substantial benefits, they also create security risks such as jailbreaks, prompt injection, and vulnerable code generation. Existing defenses...
M. Le, Nisarga Gondi, Yi-Bo Peng et al.· arXiv.org· 0 citations
CoDeL is presented, a defense that hardens agent against an attack distribution it reshapes as it trains, and reduces attack success rate (ASR) by 88.5% and outperforms other baselines largely (+38.0%).