Skip to content
Log in
Conference

Att2RAG: A Double-Condition Framework for Knowledge Poisoning Attacks on RAG Systems

Zhize Hao
2026 · Poster Volume 0008 The 2026 Twenty-Second International Conference on Intelligent Computing July 23-26, 2026 Toronto, Canada · 0 citations

TL;DR

Att2RAG is presented, a double-condition framework for knowledge poisoning attacks on RAG systems that decomposes a successful poisoning event into a retrieval condition and a generation condition, and casts poisoning as maximizing attack success subject to satisfying both conditions.

Abstract

Modern retrieval-augmented generation (RAG) and memoryaugmented LLM applications are widely deployed in knowledge-intensive settings. These systems ground model outputs on external knowledge stores and may persist interaction traces in vector memory. If the underlying store is compromised, poisoned content can be retrieved repeatedly and thereby shape downstream responses, yielding confident yet harmful outputs supported by seemingly plausible evidence. Recent studies have shown that RAG pipelines and LLM agents are vulnerable to knowledge poisoning and prompt injection, but many formulations treat attack success as a single end-to-end outcome and do not separate retrieval and generation failure modes in a retrieval–generation aligned manner. Moreover, although long-horizon memory writes can introduce cumulative risks, these effects are not empirically evaluated under our benchmark setting. We present Att2RAG, a double-condition framework for knowledge poisoning attacks on RAG systems. Att2RAG decomposes a successful poisoning event into a retrieval condition and a generation condition, and casts poisoning as maximizing attack success subject to satisfying both conditions. We instantiate the framework with (i) a white-box variant that applies projected gradient descent (PGD) in embedding space, serving as an approximate upper bound under strong attacker assumptions, and (ii) a black-box attack based on a Q ⊕ I construction that combines problem self-similarity with adversarial instruction injection using only query access. We evaluate Att2RAG on representative QA benchmarks under several common RAG configurations and consider practical defenses such as paraphrasing and duplicate filtering. Attack success is measured by ASR and Target-F1 relative to an attacker-specified target response. Across the evaluated configurations, Att2RAG attains attack success rates in the mid-90% range without defenses; paraphrasing plus duplicate filtering reduce ASR only modestly, and many attacks remain successful. These results highlight limitations of semantic-similarity-driven retrieval and suggest that strengthening RAG systems requires defenses beyond surface-form rewriting and naive duplicate removal.

DOI Semantic Scholar
View source

Similar papers

Preprint Jul 2026

RAGuard: A Layered Defense Framework for Retrieval-Augmented Generation Systems Against Data Poisoning

RAGuard, a layered defense against corpus-poisoning attacks on RAG pipelines, is introduced, showing that keyword-preserving poisons leave lexical retrievers such as BM25 essentially unaffected, an observation that delineates the boundary of the threat model.

Pushkal Kumar, Tucker Nielson, Tanish Kolhe et al. · 4 citations · ⚡1
Preprint Aug 2026

InjecMEM: Memory Injection Attack on LLM Agent Memory Systems

This work proposes InjecMEM, a novel memory injection attack paradigm that requires only a single interaction to steer later responses of related queries toward a pre-specified output and achieves reliable topic-conditioned retrieval and targeted generation.

Hanling Tian, Gengyu Zhang, Zeyang Sha et al. · 1 citation
Preprint Aug 2026

MAFIA: Query-Only Memory Attacks via Probing and Factual Injection against Audited LLM Agents

Memory-augmented LLM agents rely on rich context for long-horizon reasoning and acting, yet their memory modules expose a persistent attack surface for malicious records, making the study of memory poisoning threats imperative. However, existing query-only attacks often fail to remain effective in two realistic and prevalent settings: large-scale benign memory pools and active input auditing. Consequently, current approaches fall short when facing the dual challenges of high retrieval competitiveness and rigorous semantic checks. To overcome these limitations, we propose MAFIA, a query-only Memory Attack framework via probing and Factual Injection against Audit, tailored to this extended threat model. Specifically, MAFIA introduces: (1) a placement strategy that ensures retrieval-competitive injection via memory probing, budget allocation, and scheduling; and (2) a payload design that bypasses audits using compact factual cloaks, preserving malicious effects while maintaining high semantic similarity. Extensive evaluations reveal that MAFIA achieves up to a 90.7% attack success rate while suppressing audit detection from a peak of 83.3% to at most 7.4%, exposing critical vulnerabilities across agentic memory systems. Code will be made publicly available at https://github.com/JiamingChen1234/MAFIA.

Jiamin Chen, Yisen Gao, Yanping Li et al. · 0 citations
Preprint Aug 2026

Utility Under Attack: Agent Memory Poisoning and the Limits of Content Screening and Provenance Ranking

Persistent memory makes false information durable: once a false statement is stored, it can be retrieved into future sessions that match it. We measure the cost of this failure mode using plainly worded false assertions generated in a single pass, with no instruction, trigger, or retriever optimization. Poisoning 1.2% of a LongMemEval corpus reduces accuracy from 0.850 to 0.300. A four-stage write-time screening pipeline that reaches 0.832 recall on indirect prompt injection while flagging 1.5% of trigger-word-laden benign text rejects 0 of 360 poisoned memories. We argue this exposes a boundary of content-only screening: distinguishing a false assertion from a true one generally requires external grounding beyond the text itself. We then evaluate provenance-weighted retrieval. The shipped weight is statistically indistinguishable from no defense (p=0.80), while a stronger weight recovers utility only by excluding untrusted content. In a mixed-provenance corpus where untrusted content is mostly benign, accuracy rises from 0.3167 to 0.7000; when the answer-bearing evidence itself arrives untrusted, evidence recall falls to zero and accuracy to 0.0417. Under the measured similarity regime, the additive provenance term has no usable setting: a weight strong enough to resist query-shaped poison is also strong enough to suppress legitimate untrusted evidence. We therefore argue for bounded occupancy constraints at retrieval rather than additive provenance penalties, and release the harnesses, corpora, and aggregate run reports.

Arulnidhi Karunanidhi · 0 citations