Skip to content

Language Is a Security Surface for Agent Guards

Oct 2026 · 0 citations · 24 references
Computer Science

TL;DR

This work builds LinguaGuardBench from the 1,220 labelled steps of TS-Bench AgentDojo-Traj, translating only the user instruction into German, French, Turkish, Swahili and Amharic, and keeps every tool call, observation and label byte-identical, so any change in a verdict is attributable to language alone.

Abstract

Step-level guards decide, before each tool call, whether an agent's next action is safe given the user's request and the trajectory so far. Users write those requests in many languages, while guard prompts and the benchmarks on which purpose-built guards are trained and evaluated are English. We ask whether a guard's verdict depends on the language of the request. We build LinguaGuardBench from the 1,220 labelled steps of TS-Bench AgentDojo-Traj: we translate only the user instruction into German, French, Turkish, Swahili and Amharic, and keep every tool call, observation and label byte-identical, so any change in a verdict is attributable to language alone. Six English rewordings and a whitespace placebo serve as controls, and managed guards are run three times to measure their noise. Across 19 guards, from frontier APIs to 4B open models and four purpose-built agent guards, most open, small and purpose-built guards change verdicts under translation beyond their own noise floor, and the effect grows with the rarity of the language: among the nine guards below 13B parameters, the mean placebo-adjusted flip rate rises from +1.5 pp on French to +4.4 pp on Amharic. Purpose-built guards are the most sensitive. The direction of the damage tends to follow guard size: open guards of 12-30B parameters block more benign steps (up to +5.0 pp false positives), and every significant loss of recall is in a guard below 10B (up to -7.4 pp). Frontier guards shift by at most about one point; the strongest, Gemini 3.8 Flash, returns identical verdicts with and without the request. The damage is concentrated in a few instructions and, in a post-hoc analysis, falls on steps where the agent's reasoning restates the English request, consistent with guards reacting to inconsistency between channels. English paraphrase moves every guard less than translation does. We release the benchmark and all 489k verdicts.

View source

Similar papers

Preprint Oct 2026

ToolRosetta: Surface-Form Invariance of Tool-Call Guards

Step-level guards for tool-using agents are evaluated on tidy JSON tool calls, but deployed coding agents act through execute_bash and execute_code tools whose payloads are strings. We introduce ToolRosetta, which re-renders every candidate action in the AgentDojo and AgentHarm splits of TS-Bench as a Python call, as a...

Hasan Demirkiran, Jens Ernstberger · 0 citations
Preprint Sep 2026

Control-Token Injection Suppresses Chain-of-Thought and Defeats Reasoning-Based Oversight in Tool-Using Agents

The safety of a tool-using language model agent is usually treated as a property of the model alone. We give controlled, full-precision evidence that it is instead a joint property of the model and the software that renders its chat template and parses its tool calls, the decoding harness, and that both halves are atta...

Muhammad Usama, Khairatul Nisa, Summer Yeoreum Jung · 0 citations
#artificial intelligence Preprint Oct 2026

Shortcut Sensitivity in Agent-Safety Evaluations

AI agents take actions through tools. Some actions are sensitive, so guards that supervise them must themselves be evaluated reliably. Yet the data used to evaluate a guard can include cues that make labels predictable without measuring the intended safety construct, and those cues can also influence guard decisions. W...

Hasan Demirkiran, Jens Ernstberger · 0 citations
#artificial intelligence Review Oct 2026

Same Operation, Different Permission: Evaluating Whether Guardrails Honor Authorization

As language-model agents write code, move money and change records on behalf of users, deciding whether an action is authorized has become a central safety problem. Guardrails are often placed in front of these actions, yet few evaluations ask whether their decisions follow authorization: blocking unauthorized actions...

Hui-Hao Jing, Shao-Jin Chen, Wen-Bin Hu et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Benchmarking Jailbreak Guardrails for Embodied Agents

Embodied agents powered by large language models and vision-language models are increasingly deployed in physical environments, but jailbreak attacks can induce these agents to perform physically harmful actions. A growing number of guardrail methods have been proposed to intercept dangerous behavior before it is execu...

Xun-Guang Wang, Qing-Yue Wang, Yu-Guang Zhou et al. · 0 citations
Open access Sep 2026

Who Guards the Guard? Evaluating Deterministic, LLM-Based, and Hybrid Firewalls for Tool-Using Agents

Tool-using large language model (LLM) agents process untrusted files, URLs, and tables at a boundary where data may be mistaken for instructions. This paper compares four interface-level conditions: no firewall, deterministic policy enforcement, an LLM-based Tool-Input Minimizer/Tool-Output Sanitizer, and a gated hybri...

C. Quincozes, Paulo S. Souza, Rafael D. Araújo et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.