SemGuard: A Triple-Anchor Semantic Security Gateway for Multilingual Prompt Attack Detection in Large Language Models

Abstract

The threat posed by adversarial prompts to large language models is becoming harder to ignore. Problems including prompt injection, jailbreaking, phishing, and Unicode-based attacks are now widespread. Most existing solutions protect against only one threat type, operate in English only, and provide no explanation for their decisions. We present SemGuard, a multilingual security gateway using Triple-Anchor Semantic Threat Modeling, which simultaneously evaluates each input against three semantic reference sets: attack, safe, and destructive. SemGuard detects four threat types concurrently in Arabic, Arabizi, and English. We expand the original Arabic Security Dataset from 319 to 807 validated examples across seven threat categories, using three independent LLM judges (GPT-4o, Grok-4, Llama 3.3 70B) achieving Fleiss' $\kappa=0.839$. After retraining on the expanded dataset, SemGuard achieves a mean F1-score of 0.989 and recall of 0.991, representing a 13.7% improvement over the original implementation. Analysis of 527 rejected examples reveals quantitative evidence of threat-category ambiguity, with impersonation exhibiting a 98.2% inter-judge disagreement rate, validating the necessity of the Triple-Anchor framework. This work also presents the first Arabic LLM security dataset with a formal LLM-as-Judge annotation protocol.