Skip to content
Preprint

ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents

Aug 2026 · 0 citations · 29 references
Computer Science

TL;DR

This work defines behavioral safety and introduces ActBench, a self-evolving benchmark that evaluates such behavior risk from execution trajectories rather than final responses, and proposes a dual evidence verification mechanism that verifies agent execution safety and utility through log evidence and LLM-based trajectory evidence.

Abstract

Cowork agents may complete benign tasks while disclosing protected data, manipulating unauthorized state, invocate unauthorized API. We define behavioral safety and introduce ActBench, a self-evolving benchmark that evaluates such behavior risk from execution trajectories rather than final responses. Each case pairs a benign task with an adversarial variant that preserves its instruction, configuration, initial state, rating model, and trusted records while injecting a task-reachable payload. ActBench contains 600 cases from 213 scenarios, spanning 15 risk behaviors, six execution spaces, and 48 web-service APIs.To move beyond static payloads, we propose a reward-guided beam search method that jointly optimizes attack effectiveness and task utility, while reflection diagnoses failed execution checkpoint and guides payload revision. Besides, we propose a dual evidence verification mechanism that verifies agent execution safety and utility through log evidence and LLM-based trajectory evidence.We evaluate 15 LLMs and 6 open-source cowork agents over 24,000 trajectories. Under a fixed harness, attack success rates ranges from 10.1% to 94.4% across models, while under a fixed base model, they range from 73.7% to 94.4% across agents.These results show greater variation across models than agent harness, while attacks remain highly successful across all tested harnesses.Our benchmark is released at: https://github.com/zjuicsr/ActBench.

View source

Similar papers

Preprint Oct 2026

ASPIRE: Agentic Safety&Prompt Injection Red-teaming Engine

LLM agents retrieve untrusted content and act through tools, creating indirect prompt-injection risks that can cause unauthorized actions or persistent state changes. Existing automated red-teaming largely optimizes payloads for pre-specified scenarios, leaving latent vulnerabilities across the agent's behavior space u...

Peng-Fei He, Deepanjan Mitra, Vishesh Sharma et al. · 0 citations
#artificial intelligence Preprint Oct 2026

EvoRiskBench: An Evolving Benchmark for Runtime Security Risks in Workspace Agents

Workspace agents combine large language models with execution harnesses to perform stateful, multi-step tasks that access or modify external resources. Existing benchmarks leave gaps in executable coverage of their runtime security risks, while evolving model capabilities, harnesses, tools, and threats motivate benchma...

Shi Kuang, Xue-Mei Luo, Kun Liu et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Self-Healing Harness for Runtime Oversight of Agent Self-Modification

Across 16 matched Baseline and Harness runs spanning AppWorld, Terminal-Bench, and $\tau^2$-Bench, the gate rejected 383 replay-decided proposals, showing that locally beneficial self-modifications can introduce collateral regressions often enough to materially affect gate decisions, providing direct empirical motivati...

Sina Tayebati, Divake Kumar, Nastaran Darabi et al. · 0 citations
#artificial intelligence Preprint Sep 2026

SelfOp: An Optimization Algorithm for Self-Improving Security Agents

LLM agents are increasingly used for security tasks: vulnerability discovery, exploit reproduction, and patch generation. Improving them at the model level demands expert demonstrations or computable rewards, which security tasks rarely offer: traces are costly, failures hard to diagnose, rewards sparse, and non-comput...

Saad Ullah, Yiğitcan Kaya, Christopher Kruegel et al. · 0 citations
#artificial intelligence Review Sep 2026

Safety in Self-Evolving Agents: A Survey

SAVER is introduced, a transition-centered framework in which Substrate locates reusable influence, Adaptation captures how it changes, Violation identifies compromised safety attributes, Exposure marks where failures become observable, and Response assesses containment, repair, or revocation.

Jia-Hao Chen, Zhou Feng, Ou-Bo Ma et al. · 1 citation
Preprint Oct 2026

Compromise Is Not Consequence: Evaluating Task-Scoped Authorization in LLM Agents with Paired Replay

A tool-using model can follow a malicious instruction even when its credentials are valid. We study whether task-scoped authorization contains the resulting tool execution. Our paired-replay testbed samples a model request once and submits the same action, resource, and arguments to broad bearer, scoped JWT, sender-con...

Tural Hagverdiyev · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.