Prompt Pipelines as Engineered Artifacts: Regression-Testing a Multi-agent Threat-Modeling Pipeline
TL;DR
The experience suggests that the PromptOps practices recently proposed for general LLM applications transfer directly to security engineering, and that a fixed evaluation corpus with cost-aware metrics is the single most valuable artifact a prompt-pipeline team can maintain.
Abstract
Security pipelines built on large language models (LLMs) are, at their core, collections of prompts: role instructions, retrieval queries, critique templates, and output formats. In practice these prompts evolve the way early software did: by trial and error, with changes that are rarely documented and never regression-tested. This paper reports engineering experience from treating a multi-agent threat-modeling pipeline as a set of versioned, tested software artifacts. The pipeline decomposes STRIDE analysis into six specialised agent prompts with retrieval enrichment and a critique stage; a fixed corpus of 24 system architectures with expert-curated reference threats serves as a regression oracle for every change to a prompt or pipeline stage. We describe three documented design decisions the harness drove: removing a chunk-filtered retrieval stage that consumed tokens without improving coverage, declining to adopt the most expensive pipeline as the default after the oracle showed a six-agent decomposition matched its coverage at roughly one-ninth the token cost, and catching a metric-specification drift between documentation and implementation. We distil the recurring failure modes (retrieval-induced specificity bias, mechanism-versus-impact category confusion, and systematic risk over-estimation) into checks that now guard the pipeline. The experience suggests that the PromptOps practices recently proposed for general LLM applications transfer directly to security engineering, and that a fixed evaluation corpus with cost-aware metrics is the single most valuable artifact a prompt-pipeline team can maintain.
