The trade-off across both planes is that the less an operator can commit to in advance, the less deterministic the resulting guarantee, down to asking a human.
Abstract
AI agents increasingly sign Git commits, certify documents, and attest release artifacts on behalf of their operators, using private keys that live in software-accessible locations (plaintext files, environment variables, container memory) readable by any process the agent can reach. A widely deployed agent framework recently leaked its keys this way to a single email injection. Hardware keystores (HSM, TPM, smart card) keep the key on-device, but exposing the keystore as a tool an LLM agent can call moves the problem rather than removing it: once a signing session exists, the hardware cannot tell a request reflecting the operator's intent from one injected into content the agent read. We characterize this confused-deputy problem and build the five-layer Zero-Trust enforcement stack it requires, so that only requests consistent with the operator's committed intent reach the hardware. We evaluate on two attack planes. Prompt injection in content the agent reads (AgentDojo, three injection-following models, n=144) falls from an 18.1% baseline attack success rate to 0% under the full stack. Tool poisoning by a compromised MCP server (MCPTox) is contained identically: a hash comparison protects a pre-committed payload, and human-in-the-loop escalation contains autonomous requests with nothing pre-committed. A further probe delineates how far the semantic filter's protection extends: it detects a substitute document under an unrelated name, but an adversarially plausible substitute name defeats it in every trial we ran. We report this as a central finding: the architecture's guarantee never rests on the filter being right, only on a human being asked whenever nothing was committed in advance. The trade-off we characterize across both planes is that the less an operator can commit to in advance, the less deterministic the resulting guarantee, down to asking a human.
This paper evaluated Niyam-AI on 2,000 real-world agent scenarios from Agent-SafetyBench and compared it against three existing safety approaches: NeMo Guardrails, Meta's Llama Prompt Guard 2, and OpenAI's GPT-OSS-Safeguard.
This work argues that agents need an operating-system substrate providing mandatory, non-bypassable services for identity, input mediation, memory governance, and execution control, and introduces AgentKernel, a trust-native agent operating system built around the premise that security must be a first-class design cons...
Zhen-Hua Zou, Sheng Guo, Qiu-Yang Zhan et al.· 0 citations
KITA is presented, a review-to-authorization architecture that keeps the user's personal secret signing key and every threshold signing-key share outside all LLM processes and establishes execution-bound authorization integrity.
Agentic Provenance (AgentProv), the first action-based identity audit for agentic LLM APIs, is introduced: AgentProv fingerprints a deployed model through its categorical tool-call distribution and decides identity via an MMD permutation test.
Xun Wang, Bihe Zhao, Michael Backes et al.· 1 citation
It is argued that this gap leaves an irreducible residual of undetected attacks wherever inputs and actions are not restricted in advance to an enumerated set, and systematize defenses across runtime monitoring, architectural separation, and authorization.
NiyamAI is presented, an intent bound runtime guardrail architecture providing cryptographically verifiable execution integrity for autonomous agents and subjects its own enforcement mechanism to 18 adversarial vectors across six classes, disclosing two implementation vulnerabilities identified and remediated during de...
Aditya Katkar, Om Karkele, Kartik Mandhane et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.