Skip to content

Capability-Mediated Perimeters for Secure AI Agent Tool Execution: Conditional Non-Escalation Invariants and Empirical Evaluation Against Indirect Prompt Injection

Sep 2026 · Zenodo (CERN European Organization for Nuclear Research)
Adversarial Robustness in Machine Learning

Abstract

Abstract — Autonomous artificial intelligence agents executing over extensible tool interfaces (such as Anthropic's Model Context Protocol) operate with ambient authority over connected tools. Because autoregressive Transformers ingest instructions and untrusted third-party data within a single homogeneous context window, adversarial observations can manipulate the model into executing unintended privileged actions—the classic Confused Deputy problem. In this paper, we explore an architectural defense-in-depth approach that treats LLM agents as potentially compromised, untrusted principals. Rather than relying on linguistic moderation alone, tool dispatch is governed by an external capability-mediated reference monitor enforcing complete mediation, least privilege, and four typed relational argument invariants (destination containment, scope boundedness, privilege monotonicity, and aggregate monetary clamping). Under complete mediation axioms (A1–A6) over the trusted computing base, out-of-scope tool invocation is deterministically rejected at the transport boundary, formalized as an Inductive Multi-Step Tool Chain Composability Invariant (Proposition 1) showing that adversarial observations cannot synthesize authority across arbitrary execution sequences. However, in-scope parameter poisoning within authorized tools and cross-tool data exfiltration present harder challenges: semantic neural validation is distribution-bounded (exhibiting an empirical false-negative rate of 21.5% on in-scope manipulations prior to boundary sharpening and remaining susceptible to adversarial optimization), while cross-tool exfiltration requires explicit decentralized information-flow tracking (DIFC). We further introduce declarative stateful workflow authorization, which constrains specified multi-step action sequences as a restrictive intersection with CBAC and DIFC, and validate the integrated gateway through 23 author-constructed adversarial workflow tests covering trajectory, concurrency, desynchronization, execution uncertainty, and receipt-integrity attacks. We report empirical evaluations across both foundational baseline studies (a 50,000-sample macro benchmark and a 3,000-case ablation matrix) and a Six-Regime Empirical Validation Program totaling 6,662 evaluation cases with frozen checkpoint V6: achieving 100% Correct Identification Rate on internal factorized diagnostics (N = 145), 100% accuracy on a pre-sealed holdout suite (75/75, SHA-256: 22bc736c...), 98.43% defense on the 4,216-instance InjecAgent evaluation (2,075/2,108 attacks blocked, 1,916/2,108 benign allowed) (P50: 267.7 ms), 92.44% defense on AI Safety Bench (416/450 attacks blocked, 550/550 benign operations allowed), 99.52% attack defense on interactive AgentDojo (N = 629) with exact clean-task utility parity (6/97 tasks) matching the unprotected base agent, and 100% defense across 500 targeted adaptive red-team trials. Finally, we systemize the runtime into the Mastyf Security Gateway: the evaluated research runtime was v0.1.0-RC1 (verifying 38/38 security invariant tests); the hardened commercial-pilot runtime is v0.1.1-rc1 (verifying 118/118 tests across unit, integration, and adversarial suites), establishing complete mediation non-executability (Decision ∈ {BLOCK, ESCALATE} ⇒ BackendToolInvocations = 0) and >330,000 req/s reference monitor throughput under an Ed25519-signed release manifest. We contextualize Mastyf as an empirically evaluated pre-production architecture, highlighting residual risks and outlining requirements for broader production-scale validation.

View source

Similar papers

#artificial intelligence Open access May 2023

Evaluating the Performance of Large Language Models on GAOKAO Benchmark

GAOKAO-Bench is introduced, an intuitive benchmark that employs questions from the Chinese GAOKAO examination as test samples, including both subjective and objective questions that contribute a robust evaluation benchmark for future large language models and offers valuable insights into the advantages and limitations of such models.

Xiaotian Zhang, Chun-yan Li, Yi Zong et al. · 216 citations · ⚡17

PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selection

Empirically, PRISM reduces the end-to-end time for data selection and model tuning to just 30% of conventional pipelines, and achieves this efficiency while simultaneously enhancing performance, surpassing models fine-tuned on the full dataset across eight multimodal and three language understanding benchmarks.

Jinhe Bi, Yifan Wang, Danqi Yan et al. · 73 citations · ⚡4
#artificial intelligence Conference Open access Apr 2020

ECCOLA - a Method for Implementing Ethically Aligned AI Systems

The method, ECCOLA, is presented, which aims at making the high-level AI ethics principles more practical, making it possible for developers to more easily implement them in practice.

Ville Vakkuri, Kai-Kristian Kemell, P. Abrahamsson · 64 citations · ⚡6

Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets

This paper designs Markov decision processes (MDPs) for different combinatorial problems and proposes to train conditional GFlowNets to sample from the solution space and demonstrates that GFlowNet policies can efficiently find high-quality solutions.

Dinghuai Zhang, H. Dai, Esmeralda S. Whitammer et al. · 59 citations · ⚡8

Ethically Aligned Design of Autonomous Systems: Industry viewpoint and an empirical study

An empirical study on the current state of practice in artificial intelligence ethics is conducted by means of a multiple case study of five case companies, which indicates a gap between research and practice in the area.

Ville Vakkuri, Kai-Kristian Kemell, Joni Kultanen et al. · 56 citations · ⚡6
#artificial intelligence Conference Open access Jun 2018

The Key Concepts of Ethics of Artificial Intelligence

It is suggested that the focus on finding keywords is the first step in guiding and providing direction for future research in the AI ethics field.

Ville Vakkuri, P. Abrahamsson · 39 citations · ⚡2

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.