This paper proposes TRACE, which simulates harmful SFT trajectories to produce progressively corrupted model states, and optimizes a safety patch simultaneously across these states, and improves the safety rate by up to 77 percentage points over the best baselines, while maintaining comparable utility to the undefended...
AutoDojo is introduced, a generative benchmark built on AgentDojo and AgentDyn that generates IPI adaptively for a given agent and defense, and it is demonstrated that standard static benchmarks often significantly overestimate defense efficacy.
Xinhang Ma, Taoran Li, Chao-Wei Xiao et al.· 5 citations
To study this phenomenon, SocioHack is introduced, a sandbox of 72 societal environments, and it is found that within these environments, reward hacking naturally emerges and leads to regulatory loophole discovery.
Wei Liu, Xinyi Mou, Hanqi Yan et al.· arXiv.org· 3 citations
MemPoison is proposed, a novel memory poisoning attack that bypasses selective memory mechanisms in LLM agents, where an attacker can inject triggerable backdoors into the agent's long-term memory through dialogue interactions, thereby misleading its subsequent responses.
Hongtao Wang, Sean Yang, Yu Chen et al.· 5 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
The Infinite Impostor is introduced, an attack model in which an autonomous agent interposes itself between two parties who already trust each other, hijacking an existing relationship rather than building a new one from scratch, and the governance tensions that arise when platforms become the regulatory substrate of d...
Osama Zafar, Alexander Nemecek, Erman Ayday· arXiv.org· 0 citations
AI agents increasingly act autonomously in the world, yet harmful behavior cannot be reliably traced to the account that deployed the agent. This creates an accountability gap across both benign and malicious settings: misconfigured or hijacked agents may cause unintended harm, while malicious operators may deploy agen...
Ruben Chocron, Doron Jonathan Ben Chayim, Eyal Lenga et al.· 0 citations
It is shown that even zero-bit watermarking supports internal attribution under per-entity multi-key deployments without explicitly encoding identity, and external identification in selected text and image configurations is demonstrated.
Toluwani Aremu, Nils Lukas, Jie Zhang· arXiv.org· 1 citation
With LLM watermarking already being deployed commercially, practical applications increasingly require multibit watermarks that encode more complex payloads, such as user IDs or timestamps, into the generated text. In this work, we propose a fundamentally new approach for multibit watermarking: introducing binomial enc...
Thibaud Gloaguen, Robin Staab, Mark Vero et al.· 0 citations
FraudBench is a multimodal benchmark for detecting AI-generated fraudulent refund evidence and shows that current MLLMs often recognize real-damaged evidence but fail on many fake-damaged subsets, with fake-damage detection rates far below the 50\% baseline on most generator subsets.
Experiments show that turn-level boundary supervision improves intervention localization, while reinforcement learning further improves the safety--utility trade-off.
Xin-Jie Shen, Rong-Zhe Wei, Pei-Zhi Niu et al.· 0 citations
MOSAIC-Bench (Malicious Objectives Sequenced As Innocuous Compliance), a benchmark of 199 three-stage attack chains paired with deterministic exploit oracles on deployed software substrates that treats both exploit ground truth and downstream reviewer protocol as first-class evaluation axes, is introduced.
Research Manager Mike Walker reflects on a career spanning hacker culture, cybersecurity, and AI, sharing lessons about trust, teamwork, leadership, and why solving difficult problems requires curiosity, judgment, agility—and an understanding of the rules. The post Learn the rules: The hacker mindset appeared first on Microsoft Research.
Critical flaws, compromised release channels and agentic AI threats landed within days of each other in mid-October 2026. Why “Black October” may say more about how well we now find weaknesses than about the attacks themselves. The post Black October? The Week AI Changed the Cybersecurity Conversation appeared first on GPT-Lab.
