Jul 2026· International Conference on Smart Communications and Networking· pp. 1-6· 0 citations· 16 references
Abstract
Prompt injection poses a significant security risk to Retrieval-Augmented Generation (RAG) systems, enabling adversaries to embed malicious instructions within retrieved documents and hijack model behavior to exfiltrate sensitive information or execute unauthorized actions. This work presents a modular dynamic evaluation environment for systematically testing and comparing defense mechanisms against prompt injection attacks in RAG architectures. The framework simulates diverse injection scenarios targeting the retrieval pipeline, integrates optional mitigation strategies such as input filtering, prompt rewriting, and retrieval-aware defenses, and automatically logs model behavior to assess attack success. By varying retrieval parameters and quantifying defense robustness across diverse attack verticals and RAG configurations, the system enables reproducible and scalable evaluation of prompt injection resilience. The results highlight strengths and weaknesses of existing defenses in RAGspecific threat models and establish a foundation for standardized benchmarking of defense mechanisms in knowledge-grounded generative AI systems.
Prompt injection attacks on Large Language Model (LLM) agents seek to introduce malicious instructions or content into external text sources retrieved by agents, forcing the underlying LLMs to execute harmful actions outside their benign scope. While current defenses effectively counter known injection attacks, deploying them in LLM agent environments remains challenging due to attack variants and emerging threats. Moreover, existing solutions typically suffer from an inherent trilemma, i.e., a constant trade-off among runtime efficiency, contextual precision, and adaptability. To bridge this gap, we propose Continuous Agents for Injection Threats via Lifelong Yielding Nexus (CAITLYN), an agent-agnostic defense middleware. CAITLYN integrates two systems. System I focuses on immediate defense against existing attacks using a two-tiered library: Tier-0 for rule-based detection scripts and Tier-1 for optimized LLM-based accurate inference. System II, in contrast, is deployed to monitor potential abnormal signals and attempt to synthesize new defenses. On standard benchmarks, CAITLYN matches the detection performance of state-of-the-art defenses at lower token overhead than LLM-as-a-judge baselines. On Emerging, our new delivery-aware benchmark featuring novel injection techniques, static baselines and the standalone System I configuration remain vulnerable. In contrast, System II autonomously synthesizes verified defense capabilities, substantially lowering the attack success rate across three diverse agent environments.
Zi Liang, XiaoYu Xu, Yanyun Wang et al.· 0 citations
A systematic review and structured descriptive synthesis of research on defenses against prompt-based attacks in language model and agent systems reveals trade-offs between security effectiveness, performance, and system complexity as well as major gaps in benchmarks, indirect attack coverage, and multi-agent evaluation.
Sana Mourad, E. Abdallah, Mohammad Ababneh· Electronics· 0 citations
This paper proposes SecureMCP, a policy-enforced framework that integrates Role-Based Access Control with an MCP server to establish multi-layer defense for LLM-generated SQL execution, and evaluates filter performance—false positive rate (FPR) and false negative rate (FNR))—separately from LLM generation quality.
Wonbae Kim, Hee-Kyong Yoo, Nammee Moon· Applied Sciences· 0 citations
The results establish that LLM-based log analysis creates an inherent confused deputy vulnerability where untrusted data and trusted instructions compete indistinguishably for model attention, requiring defense in-depth architectures and continued human oversight for security-critical decisions.
Rabimba Karanjai, Yang Lu, H. Madhavarao et al.· 2 citations
Large language model (LLM) agents integrated with external tools are vulnerable to indirect prompt injections embedded in environmental states. However, existing studies largely rely on manually implemented or reused environments, stochastic LLM-based tool simulation, and predefined injection locations, limiting scalable security research across broader domains. To bridge this gap, we propose **ToolHazard**, a scalable adversarial environment synthesis framework that reduces human engineering and supports expansion with additional seed domains and compute. Through an Environment Simulator, an Attacker Agent, and a User Simulator, ToolHazard synthesizes executable stateful environments, discovers viable injection points and generates environment-specific payloads, and constructs state-grounded long-horizon tasks. Based on ToolHazard, we build **ToolHazard-Bench** for stress-testing agents under complex workflows and diverse environmental attacks. Experiments reveal substantial agent vulnerabilities and show that injection timing and placement affect attack effectiveness. Moreover, ToolHazard-generated alignment data improves security on both ToolHazard-Bench and AgentDojo while preserving benign task utility.
Yutao Mou, Pengfei Yang, Zhenfei Yin et al.· 0 citations
Compositional Attack Path Scoring (CAPS), a framework engineered to quantify end-to-end multi-hop risks in LLM architectures, establishes a rigorous benchmark for quantitative vulnerability management in complex, agentic LLM environments.