AI agents are demonstrating increasingly strong offensive and defensive cybersecurity capabilities in the real world. Current benchmarks mostly evaluate agents in isolation against static targets. Real cyber operations, however, are dynamic and coupled: security postures constantly change as attackers and defenders ada...
Bangya Liu, Jianhong Tu, Yu-Jing Pan et al.· 0 citations
CyberArena is a dynamic, symmetric adversarial benchmark built around multi-round attack-and-defense combat, and introduces a dynamical model that maps match dynamics into unified, multidimensional scores across offensive, defensive, and domain-specific capabilities.
Bangya Liu, Jianhong Tu, Yu-Jing Pan et al.· 0 citations
This work presents AgentXploit, a two-role auditing system that separates repository-level attack-path discovery from runtime exploitation and introduces AgentXploit-Bench, containing 72 reproducible vulnerabilities across 12 open-source AI-agent systems and frameworks.
Wei-Da Liang, Shi Qiu, Zhun Wang et al.· arXiv.org· 1 citation
SafeClawArena is developed, a benchmark of 406 adversarial tasks executed in containerized replicas of real agent platforms with canary-marked credentials and evaluated via automated taint tracking across nine output channels, exposing the inadequacy of current defenses and suggesting directions for future hardening.