This work presents a trust-boundary-centric survey of foundation-model-powered embodied-agent security, and shows that attack research is concentrated on multimodal perception and action interfaces, while defenses are especially concentrated on action-level and runtime protection.
Abstract
Foundation models are increasingly used for perception, reasoning, planning, and action generation in embodied agents, creating security risks that can propagate from digital inputs to physical behavior. Existing surveys often organize threats by mechanisms such as jailbreaks, prompt injection, backdoors, poisoning, or adversarial examples, but these categories do not consistently identify where an adversary first enters the embodied control loop. We present a trust-boundary-centric survey of foundation-model-powered embodied-agent security. Using a first-compromised-trust-boundary principle, we separate attack surface from attack mechanism and organize the system into five layers and twelve attack surfaces spanning the model supply chain, user instructions, context and memory, physical semantic environments, multimodal perception, world state, internal reasoning, task planning, action interfaces, middleware, multi-agent communication, and execution control. Based on 58 attack records and 61 defense records collected through August 15, 2026, we analyze representative attacks, cross-layer propagation, defense placement, and evaluation practices. Our quantitative analysis shows that attack research is concentrated on multimodal perception and action interfaces, while defenses are especially concentrated on action-level and runtime protection. Context and long-term memory, middleware and networking, world-state integrity, and multi-agent trust remain comparatively underexplored. We conclude with open challenges in state provenance, compositional defenses, long-horizon attack propagation, physical realizability, Byzantine multi-robot behavior, and unified closed-loop evaluation.
World models give embodied AI a predictive core: they compress observations into states, simulate action-conditioned futures, and enable planning beyond reactive control. This predictive layer, however, opens a new security boundary-compromise can propagate from data, sensors, prompts, or feedback into physical action. Rather than treating world models as an isolated component, this survey traces threats across their entire lifecycle-from data construction and representation learning, through state grounding and imagination, to trajectory evaluation, execution, and long-term adaptation via memory and tools. We show that familiar attack families: poisoning, backdoors, adversarial examples, sensor spoofing, prompt injection, trajectory manipulation, and supply-chain attacks take on distinct meanings when they corrupt world states, learned dynamics, affordance estimates, or safety costs. We also highlight a duality: world models can serve as runtime safety shields, yet when compromised or over-trusted they generate predictive safety illusions. The survey offers a lifecycle taxonomy, maps existing attacks to world-model security properties, outlines evaluation protocols for safety failures, and structures defenses across provenance, robust grounding, uncertainty-aware prediction, trajectory gating, feedback auditing, and deployment assurance.
Fazhong Liu, Zhuo Chen, Haozhen Tan et al.· 3 citations
Cyber-capable AI agents combine language models with tools, memory, and execution environments to perform multi-step offensive-security tasks. Existing work separately measures cyber capability and catalogs attacks against agent components, but provides less guidance on containing a capable agent within the environments used to evaluate it. This review synthesizes five vulnerability classes at that boundary: multi-step offensive chains, objectives that conflict with sandbox boundaries, supply-chain and credential exposure, persistent command-and-control, and the speed of automated action. We use two separate preliminary incident records: the reported July 2026 Hugging Face/OpenAI evaluation breach and Anthropic's subsequent three-incident evaluation review. A comparative evidence protocol distinguishes record-specific factual claims from the shared systems lesson: the evaluation environment is itself part of the security boundary. Across the taxonomy and records, we examine controls for containment, privilege separation, provenance, and responder access, including the dual-use problem that defensive artifacts may also enable misuse. The review identifies practical priorities for evaluating cyber capability together with the security of the environment in which that capability is exercised.
Generative AI has industrialized the attack on human judgment: voice cloning, real-time video
synthesis, and hyper-personalized phishing convert social engineering from a craft into a
commodity, while a residual category of attacks — those that redefine the objective itself, as
ransomware once did — cannot be enumerated in advance. We propose a defensive architecture
for these two conditions built on a single organizing claim: under unbounded attack techniques
and unenumerable attack objectives, the highest-value security investments are those that hold
regardless of technique. The architecture layers a mesh of strictly advisory AI sentinel agents,
which detect cross-surface incoherence rather than synthetic content, above a deliberately simple,
non-interpretive enforcement core holding a small set of hard invariants that no intelligence —
human or artificial — can rewrite quickly. We give the reactive-defender objection a full
treatment: the claim that any AI-era defense necessarily responds late. Our answer is that the
objection is correct for every layer that must recognize attacks, and that the architecture is
designed around that concession — detection layers are built for their own capture, while the
invariant core does not race because it constrains consequences rather than recognizing
techniques. The framework relocates residual reactivity to a single measurable point, the interval
between the world changing and the invariant set catching up, and proposes that interval as the
headline resilience metric. We state residual risks explicitly and outline an evaluation agenda.
Ajayi Abisoye, N. Hussain, Abolaji Adebayo· International Journal of Eng...· 0 citations
Security in autonomous AI systems is not a property of individual components such as identity verification, static policy configurations, or model-level safeguards. Instead, it is a dynamic system property emerging from three interconnected dimensions: intent, time, and interaction. Intent captures the alignment between an operator’s goals and an agent’s behavior; time reflects the evolution of security states under adversarial pressure; and interaction describes how trust relationships between agents can propagate misalignment. This paper synthesizes the findings of the five-paper AI Agent Security Series into a unified, formal, and falsifiable theory of autonomous agent security. Current AI security approaches focus on component-level controls, including authentication, policy enforcement, and output filtering. However, results from Papers 1–5 show that these mechanisms are individually insufficient and collectively vulnerable to clean attacks semantically valid, policy-compliant adversarial inputs that redirect agent behavior while evading existing detection methods. We argue that such failures are structurally inevitable under component-centric security models. To address this gap, we introduce the Unified Security Function, Φ_security(φ, t, G), a scalar measure in the range [0,1] that combines IntentAlign, TemporalStability, and NetworkIntegrity into a single dynamic assessment of security health. Using this framework, we establish three meta-theorems: the Component Insufficiency Theorem, the Dynamic Necessity Theorem, and the Interaction Irreducibility Theorem. Evaluation on the complete AegisBench benchmark suite (1,560 sessions across Papers 1–5) shows that Φ_security predicts attack success with an AUC of 0.943, outperforming all individual metrics. By defining autonomous agent security as a distinct scientific object, this work provides a foundational framework for future research on secure and adaptive AI systems.
Harsh Verma· International Journal of Sci...· 0 citations
AI agents are being used more in high-pressure situations like managing email, running code, engaging with financial APIs, and supervising multi-agent pipelines. However, current taxonomy of adversarial attacks was mostly proposed for classifiers and generative models alone and fails to adequately describe the testbed of an agent with persistent state, multiple tools, and delegated power. A previously unstated class of adversarial input called a clean attack - an input that is syntactically correct, semantically consistent with the declared task context, consistent with all observable policy constraints, similar to legitimate operator instructions and still has the goal of misguiding the agent away from the original operator goal - is identified and formalized in this paper. These attacks go around the exposed dots of the “traditional” agent security architecture that only filters on the surface. The paper has three main contributions. One, it brings in a formal definition of the clean attack as a four-tuple of input, intent vector, policy envelope and behavioral outcome. Second, it suggests two operationalizable metrics: semantic validity score (SVS) and behavioral drift index (BDI) for systematically measuring the severity of clean attack. Third, the paper these metrics and taxonomy are validated, both by a purpose-built benchmark, AegisBench, and by 300 attack scenarios in three agent classes and (twelve) commercial agent pipelines. The experimental results show that clean attacks are a safety threat of a different category: while the conventional adversarial tasks are practically impervious to these attacks (4.2% success rate of the strongest agents), they achieve a mean attack success of 61.4%.
Harsh Verma· International Journal of Sci...· 0 citations
It is argued that adversarial vulnerability stems from the absence of boundary verification, a security primitive that enforces explicit validation of data as it crosses inter-agent boundaries, including content, identity, execution intent, and state integrity.
Faisal Haque Bappy, Tahrim Hossain, T. S. Zaman et al.· 0 citations
