A Gateway-Centric Architecture for Bidirectional Content Moderation in Large Language Models

Abstract

Large language model (LLM) applications deployed in regulated and high-risk environments face two coupled runtime risks: input-side prompt injection and output-side content compliance violations. Existing defenses are often either model-centric (alignment or fine-tuning) or application-local (keywords and regex), which limits auditability, policy agility, and cross-model portability. This paper proposes a gateway-centric framework that externalizes safety enforcement into a security side-chain. The framework integrates a unified LLM gateway with an external semantic guardrail service to perform bidirectional inspection before and after model invocation. We formalize the architecture with input-risk and output-compliance functions, define a comprehensive threat model and trust boundary, and present a reproducible evaluation protocol. To validate our approach, a prototype implementation based on LiteLLM and Prisma AIRS was evaluated on a rigorously balanced 2,000-sample adversarial dataset merged from diverse sources, including Deepset, wildjailbreak, and jailbreak-classification. Results demonstrate that the proposed architecture achieves an exceptional balance in detection capability, yielding an F1-score of 0.8322, a recall of 0.8531, and effectively maintaining a low False Positive Rate (FPR), thereby outperforming multiple state-of-the-art baselines including Llama Prompt Guard and Deepset DeBERTa. Furthermore, performance testing confirms practical deployability, showing an average added latency of only 320 ms under a 50-concurrent-user load. The paper concludes by discussing the side-chain advantages over model alignment mechanisms, guardrail bypass risks, and adaptive policy-evolution strategies for enterprise AI governance.