Step-level guards for tool-using agents are evaluated on tidy JSON tool calls, but deployed coding agents act through execute_bash and execute_code tools whose payloads are strings. We introduce ToolRosetta, which re-renders every candidate action in the AgentDojo and AgentHarm splits of TS-Bench as a Python call, as a...
This work builds LinguaGuardBench from the 1,220 labelled steps of TS-Bench AgentDojo-Traj, translating only the user instruction into German, French, Turkish, Swahili and Amharic, and keeps every tool call, observation and label byte-identical, so any change in a verdict is attributable to language alone.
AI agents take actions through tools. Some actions are sensitive, so guards that supervise them must themselves be evaluated reliably. Yet the data used to evaluate a guard can include cues that make labels predictable without measuring the intended safety construct, and those cues can also influence guard decisions. W...
Hasan Demirkiran, Jens Ernstberger· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
