Skip to content

MetaPI: Constructing Prompt Injection Benchmarks from LLM Agent Benchmark

Nov 2026 · 0 citations · 44 references
Computer Science

TL;DR

This paper presents MetaPI, an extensible toolkit that automatically identifies plausible injection points, generates realistic attack objectives, optimizes adaptive payloads, and validates attacks in the real environment, and enables scalable and consistent security evaluation across diverse and evolving agent ecosystems.

Abstract

Large language model (LLM) agents increasingly perform complex tasks using high-privilege tools, making indirect prompt injection (IPI) a critical security risk. However, IPI benchmarks lag behind their rapidly evolving utility counterparts, only offering limited task and environment coverage and relying on narrow, static attack scenarios. In this paper, we propose meta-benchmarking for IPI: transforming existing utility benchmarks into end-to-end IPI evaluation suites that reuse their realistic tasks, environments, and infrastructure. We present MetaPI, an extensible toolkit that automatically identifies plausible injection points, generates realistic attack objectives, optimizes adaptive payloads, and validates attacks in the real environment. Its modular interfaces support heterogeneous environments, agents, defenses, and attack algorithms, allowing new components to be integrated without redesigning the evaluation pipeline. We apply MetaPI to AppWorld, VitaBench, and Terminal-Bench and evaluate representative agents and defenses across multiple backbone LLMs. Our results show that no evaluated defense simultaneously achieves strong utility and strong security in realistic, adaptive agentic IPI settings. MetaPI thus enables scalable and consistent security evaluation across diverse and evolving agent ecosystems

View source

Similar papers

#artificial intelligence Preprint Oct 2026

INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems

LLM-based agents increasingly execute long-horizon, end-to-end workflows involving shell commands, code modification, dependency installation, and interaction with remote services. This growing autonomy expands their exposure to indirect prompt injection (IPI), where adversarial instructions embedded in external conten...

T. Nguyen, Jing-Yi Guo, Rana Seda Dubauskas et al. · 1 citation
#machine learning Preprint Nov 2026

DistDebug-Bench: Can LLM Agents Diagnose Bugs in Distributed Systems?

DistDebug-Bench, a benchmark for evaluating agents'ability to diagnose complex-system failures at the code level, curates 120 high-quality real-world bugs from 11 widely used distributed systems, spanning 5 programming languages and 10 failure patterns.

Yuxuan Jiang, Cheng Liu, John Rose et al. · 0 citations
Open access Oct 2026

CAST: A Compiler-Based Framework for Systematically Testing LLM Compositional Safety

This paper proposes CAST, a systematic testing framework designed to evaluate the compositional safety of LLMs in the domain of malicious code, and demonstrates that CAST systematically exposes severe safety violations in strongly aligned models that resist conventional red-teaming.

Lu Yan, Zhuo Zhang, Xiang-Zhe Xu et al. · 0 citations
Conference Open access Sep 2026

Hijacking Tool-Integrated LLM Agents via Indirect Prompt Injection: A Benchmark Scaffold and Threat Taxonomy

Tool-integrated large language model (LLM) agents compose privileged actions — file reads, email dispatch, shell invocation — from the reasoning traces of a probabilistic model. Because the same channel carries both task data and latent instruction, any adversary who can place text into the data pipeline gains a tool-c...

Amani M. Shatnawi, Manal Alhathli, Ola Karajeh et al. · 0 citations

Divide and Inject: Can Agents Reconstruct an Indirect Prompt Injection from Fragments?

Agentic systems are now being widely used to orchestrate tools and reason over long contexts. However, the improving capabilities of the large language models powering these agents also create new attack surfaces for indirect prompt injection. In particular, an attacker may not need to place a complete malicious instru...

Michael Lee, Zhi-Peng Wei, Yue Dong et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.