CART turns red teaming from a one-time checklist into a continuous, adaptive, and auditable search for model and agent weaknesses, suggesting that contextual adaptation can reveal weaknesses that direct prompt replay does not exercise.
Abstract
Automated red teaming often replays a fixed set of prompts, which measures known risks but cannot learn from failures found during testing. We present CART (Closed-Loop Adaptive Red Teaming), a framework that uses each result to guide what it tests next. CART begins with broad risk coverage, follows weaknesses that emerge, keeps new probes diverse, and records the evidence and source of every finding. It separates the Challenger that creates tests, the Target being tested, which may be a text-only model or a bounded tool-using agent, and the Judge that evaluates the results, allowing these roles to be studied independently. Across three evaluation families (Frontier, JAH, and Agentic), CART discovers more failures and higher average risk than static seed replay for every Target with an available baseline. The gains extend to tool-mediated agent tests, suggesting that contextual adaptation can reveal weaknesses that direct prompt replay does not exercise. These results describe what the test policies discover, not how often failures occur in real deployments. We also find that Challenger-Judge choices affect the evidence uncovered, highlighting the need for role separation and independent review. Overall, CART turns red teaming from a one-time checklist into a continuous, adaptive, and auditable search for model and agent weaknesses.
A repeat-run evaluation framework that aligns semantically equivalent categories and focuses on two operating metrics: theme churn, the normalized change in the returned category set, and volume disagreement, the change in counts for categories that persist shows that taxonomy grounding produces more consistent and rep...
This work proposes PROBE (Policy Rule Observation via Behavioral Exploration), an algorithm for training LLM-based auditors that interact with deployed agents as realistic users and probe them for policy violations through multi-turn, tool-mediated dialogue, and formalizes agent auditing as a two-player Markov game.
Kshitij Mishra, Abhijith Sharma, Nils Lukas et al.· 1 citation
KC-Bench is introduced, a controlled multi-turn benchmark for measuring model-level behavior across world-knowledge conflicts, input inconsistencies, and multi-source temporal conflicts, and provides a reproducible diagnostic for developing conflict-aware reasoning and execution safeguards.
Yaxing Lyu, Sheng-Jie Zhou, B. Toh et al.· 2 citations
The evidence supports mimeo as a compact, inspectable reference on a person, not as a demonstrated transfer of their judgment; the evidence supports mimeo as a compact, inspectable reference on a person.
Agent harnesses enable large language models to act through tools, observe the results, and iteratively work toward a task goal. Verification checks and revises agents'work to improve output quality. Yet there is limited guidance on how harnesses should organize this effort. We study verification as an axis of test-tim...
Zheng-Xin Zhang, Er-Kang Zhu, Yin Lin et al.· 0 citations
An LLM agent using tools can fail silently. It stops before the task is done, raises no error, and answers fluently. Two kinds of monitor address this, one deciding whether a run failed and the other naming the step where failure became unavoidable. Much of the reported performance on both tasks comes from regularities...