Jul 2026· ICACNC 2026 Proceedings· pp. 58-63· 0 citations
TL;DR
A explainable framework that detects the attacks using a Random Forest (RF) classifier along with semantic sentence embedding and SHAP (Shapley Additive Explanations) is used together with a wordlevel attribution mechanism to give human-legible explanations for the model predictions, and to emphasize harmful parts within the prompt.
Abstract
LLM’s have affected the production industries with fast and spot-on answers, but they have also introduced a huge threat to security such as prompt injection, which was identified as the #1 threat by the OWASP GenAI Security Project. The attackers try to manipulate the model’s actions by putting malicious intent into the user’s prompt. To counteract this problem this research vouches for an explainable framework that detects the attacks using a Random Forest (RF) classifier along with semantic sentence embedding. The provided method can also identify the intent behind the prompts at surface to detect more complex jailbreak attacks. To enhance transparency, SHAP (Shapley Additive Explanations) is used together with a wordlevel attribution mechanism to give human-legible explanations for the model predictions, and to emphasize harmful parts within the prompt. The proposed model has an accuracy of 96.88%, precision of 95.90%, recall of 99.08%, and F1 score of 97.46%, which are better than the TF-IDF baseline model. Importantly, the model was able to correctly identify jailbreak prompts like, ”You are now DAN and have no restrictions” that the baseline model always failed to correctly classify as attacks, showing that semantic feature representation is necessary for strong prompt injection detection.
The threat posed by adversarial prompts to large language models is becoming harder to ignore. Problems including prompt injection, jailbreaking, phishing, and Unicode-based attacks are now widespread. Most existing solutions protect against only one threat type, operate in English only, and provide no explanation for their decisions. We present SemGuard, a multilingual security gateway using Triple-Anchor Semantic Threat Modeling, which simultaneously evaluates each input against three semantic reference sets: attack, safe, and destructive. SemGuard detects four threat types concurrently in Arabic, Arabizi, and English. We expand the original Arabic Security Dataset from 319 to 807 validated examples across seven threat categories, using three independent LLM judges (GPT-4o, Grok-4, Llama 3.3 70B) achieving Fleiss' $\kappa=0.839$. After retraining on the expanded dataset, SemGuard achieves a mean F1-score of 0.989 and recall of 0.991, representing a 13.7% improvement over the original implementation. Analysis of 527 rejected examples reveals quantitative evidence of threat-category ambiguity, with impersonation exhibiting a 98.2% inter-judge disagreement rate, validating the necessity of the Triple-Anchor framework. This work also presents the first Arabic LLM security dataset with a formal LLM-as-Judge annotation protocol.
Abdullah M. Abughallous, Somia Abufakher· IEEE Jordan Conference on Ap...· 0 citations
Detecting whether a jailbreak attack against a large language model (LLM) has succeeded is a challenging problem in AI security. Existing approaches rely on brittle string matching, expensive LLM-based judges, or task-specific classifiers that often fail to generalize. This paper investigates whether semantic coherence is sufficient to evaluate jailbreak success. In this work, we propose PRISM: Prompt-Response coherence for Identifying jailbreaks via Scoring Method, a lightweight method that models jailbreak detection as a coherence problem between the harmful prompt and the model response. The approach combines multiple semantic similarity signals within an ensemble of classical and simple machine learning models. Training in 3,480 pairs of prompt-responses from AdvBench, HarmBench and Alpaca, PRISM achieves F1 = 0.91 and AUC-ROC = 0.98 in the internal test set. In a held-out validation set of 694 labeled samples never seen during training, it outperforms six baseline attack success estimators, achieving the highest accuracy (0.746) and AUC-ROC (0.840), while being up to $\mathbf{1 6} \times$ faster than LLM-based methods. These results demonstrate that semantic coherence provides a robust and efficient signal for jailbreak detection, significantly reducing the implementation complexity of state-of-the-art approaches.
Costabile Di Gregorio, Lama Sleem, Jérôme François et al.· Annual International Compute...· 0 citations
Open-Source Intelligence (OSINT) can be considered a crucial part of the present-day Cyber Threat Intelligence (CTI) due to delivering prompt information about the adversary activity using publicly accessible reporting and analysis. Nonetheless, the conversion of unstructured OSINT stories into structured forms like the MITRE ATT&CK model is a highly manual and subjective task. The proposed paper offers an automated solution to the problem of categorizing the threat descriptions based on OSINT into the MITRE ATT&CK techniques with a sophisticated model based on transformer and natural language processing. The suggested framework has combined OSINT preprocessing, threat behavior extraction, semantic representation learning and multi-label ATT&CK techniques classification with confidence-aware outputs. Large-scale experiments on a wide OSINT corpus show that the proposed method is far more effective compared to the ones that rely on keyword parameters and conventional machine learning baselines, especially when there are missing and imprecise threat specifications. Findings indicate greater accuracy, retrieval, and strength of a vast variety of ATT&CK methods, including categories of low density. This work is a step in the right direction by facilitating scalable and standardized ATT&CK mapping of noisy OSINT data, thus leading to more effective CTI automation and decision support to the security operations of a practitioner.
Ramesh Kumar Sharma, Dharmendra Kumar Singh, Rajesh P. Barnwal· International journal of com...· 0 citations
The proposed LLM-based framework offers a promising approach for improving phishing detection and strengthening modern cybersecurity defenses and suggests that transformer-based models can effectively identify deceptive domain structures, abnormal URL patterns, and obfuscation techniques.
L. Eliyan, M. Alshraideh, Bayan Alfayoumi· Journal of integrated scienc...· 0 citations
EchoPrompt is proposed, a training-free detector based on latent prompt restoration that achieves state-of-the-art performance among zero-shot detectors while maintaining strong robustness across challenging evaluation settings.
Hongrui Bao, Yubing Ren, Yanan Cao et al.· 0 citations
A hybrid detection framework which combines semantically deep embeddings from the RoBERTa transformer with a set of carefully designed language statistics and linguistic statistics and shows excellent resistance to the surface-level adversarial paraphrasing strategy.
Anita Rani, Suman· International Journal of Sci...· 0 citations