Jul 2026· International Journal of Information Security· Vol 25· 0 citations· 69 references
Computer Science
TL;DR
Experimental results on English and Korean SMS datasets indicate that LLMs are highly robust for spam detection and can provide interpretable rationales by leveraging their language understanding capabilities, suggesting that LLMs are a highly effective and practical option for modern SMS spam filtering.
Abstract
The widespread global use of short message service (SMS) has heightened the importance of reliable spam detection, as spam messages are not only disruptive but can also facilitate phishing and identity theft. While extensive prior research has explored SMS spam detection using traditional machine learning (ML), deep learning (DL), and transformer-based encoder models, the recent rapid advances in large language models (LLMs) with strong natural language understanding capabilities motivate a fresh, systematic evaluation. In this study, we investigate how effectively LLMs detect SMS spam using both English and Korean datasets. We benchmark representative prior approaches on recent SMS spam corpora and compare them against LLM-based classifiers. Our experiments cover both commercial black-box LLMs and locally deployable white-box models and examine multiple prompting strategies, including zero-shot, few-shot, and chain-of-thought (CoT) prompting. Furthermore, to reflect adversarial characteristics commonly used to evade detection, we evaluate robustness under concept drift and adversarial attack settings. Experimental results on English and Korean SMS datasets indicate that LLMs are highly robust for spam detection and can provide interpretable rationales by leveraging their language understanding capabilities, suggesting that LLMs are a highly effective and practical option for modern SMS spam filtering.
Despite the advancements made by researchers, spam emails remain one of the biggest challenges in the field of cybersecurity. Spam emails can serve as phishing emails or carry viruses that compromise the security of an organization's system. Current detection techniques depend on supervised learning or rely on cloud-based services, which can compromise user data privacy and affect implementation flexibility. This paper evaluates the capability of five large language models (LLMs) in zero-shot spam email classification. The models used in this study include llama3.1:8b, deepseek-r1:8b, gemma3:4b, falcon3:7b, and mistral:7b. In addition to predicting whether the email is spam or not, the LLM was also asked to generate an explanation of its prediction in natural language form. The experiments were conducted on two benchmark datasets: the Ling and TREC2007 datasets. In terms of performance, llama3.1:8b outperformed other LLMs when evaluated on the TREC2007 dataset (98.78% accuracy) and deepseek-r1:8b had the best performance on the Ling dataset (98.79%). The results show that open-weight LLMs can achieve competitive spam detection performance in a local, privacy-preserving environment without any fine-tuning.
Vusal Shahbazov· 2026 7th International Confe...· 0 citations
Email spam filtering is the process of detecting and preventing spam messages from making their way into users' inboxes while allowing valid email to be delivered. Out of various strategies used for spam detection, Long Short-Term Memory (LSTM) is possibly one of the most effective methods due to its ability to handle sequential data and model long-term temporal patterns. This paper aims to explore the state of the art in LSTM networks for email spam detection and present a systematic approach to their use. Using LSTM networks in spam detection has many advantages over traditional spam detection. They can evaluate the semantic context of e-mail content and subject lines much better, which makes them extremely useful for spam detection. Moreover, they can adapt to new types of spam as they occur, keeping them relevant and useful in changing environments. However, despite their benefits, LSTM networks face challenges with computational complexity, which needs to be addressed for better performance when training and deploying them. In the future, we may combine LSTM networks with other deep learning methods, for instance, Convolutional Neural Networks (CNNs), to enhance their ability to extract more durable features from email. Such a hybrid methodology can improve spam detection systems' ability to detect spam and their effectiveness as well as accuracy. By addressing these problems and exploring new approaches, this study aims to improve current research and application of email spam detection and strengthen security solutions in the area.
Ekramul Haque Tusher, Mohd Arfian Ismail, Nurfadhilah Idris et al.· JOIV: International Journal...· 0 citations
Experimental evaluations on the UCI SMS Spam Collection dataset demonstrate that the TF-IDF-PCA-DT pipeline achieves a detection accuracy of 99% while reducing model size by 77% and inference time by 75%, demonstrating the practicality of the proposed framework for real-world deployment in resource-constrained environments.
Baraa Tareq Hammad, Ismail Taha Ahmed, Mohamed A. Hafez et al.· Computers· 0 citations
: Spam and phishing emails remain a major cybersecurity challenge, even after decades of research into reliable detection methods. Modern ML-based spam filters typically rely on text embeddings to represent email content, yet the choice of embedding model and size is often treated as secondary. This work empirically compares a diverse set of sentence embedders to assess how model type and embedding dimensionality influence downstream email spam classification. Using both classical and ML-based classifiers, we evaluate performance across multiple embedding configurations. Our results show that embedder choice - especially embedding size - substantially affects classification performance and generalisation. We observe performance differences of up to 13% overall, alongside variations of 25% in misclassified spam and 10% in misclassified ham across embedders. These findings highlight that embedding models are not interchangeable; rather, their deliberate selection is just as critical as choosing the right classifier when designing AI-based spam detection pipelines.
Malte Josten, Gérald Kämmerer, Arne Kummerow et al.· Proceedings of the 23rd Inte...· 0 citations
Social media platforms have become valuable sources for detecting phishing due to their widespread use and rapid content sharing. Among them, Telegram stands out as an open, data-rich environment, offering access to large public groups where phishing links frequently appear. This study investigates whether Large Language Models (LLMs) can serve as effective tools for phishing detection on Telegram. We systematically evaluate three representative LLMs — GPT-4o, LLaMA-3.1, and DeepSeek V3 — across multiple prompting strategies and input configurations. For comparison, we employ an improved version of traditional feature-based machine learning models from the literature, enhanced with resampling techniques to address the strong class imbalance in our data. Experimental results show that while LLMs demonstrate some potential, achieving a Recall for the phishing class of up to 0.554 with DeepSeek — representing the model’s ability to correctly identify 55.4% of all actual phishing messages — they suffer from extremely low precision (0.065 with DeepSeek), resulting in a low F1-score of 0.114. In contrast, our feature-based models achieve a significantly more reliable precision-recall balance, with an overall phishing F1-score of 0.749 and a precision of 0.738. The findings in this study highlight the potential and current limitations of LLMs for phishing detection in dynamic instant messaging environments and emphasize the superior performance of platform-tailored models.
Md Erfan, Paula Branco, Guy-Vincent Jourdan· Digital Threats: Research an...· 0 citations
SMS spam and smishing detection is an important problem that arise in automatic message filtering systems. Already existing solutions make the use of complex deep learning models that act as black boxes. In this work, we provide a combined comparison of classical machine learning models, ensemble methods as well as deep learning approaches for classifying SMS messages mainly into three categories: ham, spam, and smishing. Classical and ensemble models use TF-IDF features to convert text into numeric vectors, whereas deep models include BiLSTM and a transformer-based BERT for semantic representation learning. The experiments are conducted on a balanced public dataset using the same preprocessing steps and evaluation process for all models to ensure a fair comparison. Both the neural models are trained for an identical number of epochs so that their comparison is not affected due to unequal training budgets. The models used in this study include Logistic Regression, Multinomial Naïve Bayes, Linear SVM, Random Forest, XGBoost, BiLSTM, and BERT. Results show that all the proposed models exhibit high classification performance, with BiLSTM having the maximum accuracy of 98.19% which is closely followed by BERT (98.14%), Linear SVM (98.04%), and Random Forest (97.99%), while simpler models remain competitive, highlighting the trade-offs between accuracy, interpretability, and computational cost. Under an equal-epoch training budget, BiLSTM and BERT achieve nearly similar performance, indicating that the two architectures are almost evenly matched. This study provides useful practical insights for building effective and efficient SMS spam and smishing detection systems.
Anurag Mukherjee, Subhas Barman, Rebanta Sarkar et al.· 2026 7th International Confe...· 0 citations