Fine-Tuning LLMs for Operational Phishing Email Detection

Can LLMs Keep Up? Evaluating Phishing Detection on Telegram

Social media platforms have become valuable sources for detecting phishing due to their widespread use and rapid content sharing. Among them, Telegram stands out as an open, data-rich environment, offering access to large public groups where phishing links frequently appear. This study investigates whether Large Language Models (LLMs) can serve as effective tools for phishing detection on Telegram. We systematically evaluate three representative LLMs — GPT-4o, LLaMA-3.1, and DeepSeek V3 — across multiple prompting strategies and input configurations. For comparison, we employ an improved version of traditional feature-based machine learning models from the literature, enhanced with resampling techniques to address the strong class imbalance in our data. Experimental results show that while LLMs demonstrate some potential, achieving a Recall for the phishing class of up to 0.554 with DeepSeek — representing the model’s ability to correctly identify 55.4% of all actual phishing messages — they suffer from extremely low precision (0.065 with DeepSeek), resulting in a low F1-score of 0.114. In contrast, our feature-based models achieve a significantly more reliable precision-recall balance, with an overall phishing F1-score of 0.749 and a precision of 0.738. The findings in this study highlight the potential and current limitations of LLMs for phishing detection in dynamic instant messaging environments and emphasize the superior performance of platform-tailored models.

Cross-model evaluation of phishing detectors against LLM-generated emails

Phishing remains a prevalent cyberattack vector, and the widespread adoption of large language models (LLMs) has enabled adversaries to generate grammatically correct and contextually coherent phishing emails at scale, against which conventional detection systems are less effective. Although stylometric methods achieve over 95% accuracy within a single generator, their performance has not been systematically evaluated when the source model changes between training and deployment. This represents a significant gap, as adversaries can switch generators rapidly. A balanced corpus of 9,986 phishing emails was assembled, comprising 4,986 emails generated by three modern LLMs (GPT-4.1, DeepSeek 3.2, and Llama 3.3 70B) across five thematic categories, and 5,000 human phishing emails sampled in a stratified manner from five public sources. Seventeen stylometric features were extracted, and Logistic Regression and XGBoost classifiers were evaluated under intra-model, cross-model, threshold-recalibrated, cross-dataset, and aggregated-pool settings. Intra-model F1 scores reached 0.96 under stratified cross-validation and 0.999 on held-out splits used for the cross-model matrix. However, cross-model F1 dropped by 28.0 percentage points under the default decision threshold of 0.5. Notably, the area under the receiver operating characteristic curve remained above 0.96 in every off-diagonal cell, indicating that discriminative information is preserved even though the decision threshold is generator-specific. Recalibrating the threshold on a small target subset reduced the gap to 4.0 percentage points (an 86% reduction), and an aggregated-pool detector achieved F1 = 0.997 on each generator. This work reframes cross-model phishing detection from a problem of model incompatibility to one of practical calibration, and provides two deployable solutions, threshold recalibration on a small target slice and aggregated-pool training, along with a publicly released multi-LLM corpus.

PiMRef: Deducing Ever-evolving Spear-phishing Emails with Knowledge Base Invariants

DeepSeek Under Attack: An Analysis of Jailbreak Attacks and Prompt-Level Defenses

The reasoning model DeepSeek-R1 was more susceptible to jailbreak attacks than the non-reasoning model DeepSeek-V3, and different prompt-level defenses were effective against static jailbreaks.

Authenticity Detection by Fine-Tuning Transformer-Based Models

The rapid spread of AI-generated misinformation poses serious risks to public discourse and societal stability, creating an urgent need for automated fake text detection systems. This paper presents an authenticity detection framework that integrates transformer-based architectures (RoBERTa and BERT) with parameter-efficient fine-tuning strategies, specifically DoRA and LoRA, evaluated on the FakeNewsNet dataset. The proposed RoBERTa + DoRA model achieves 88.40% accuracy on FakeNewsNet while updating less than 1% of model parameters, outperforming full fine-tuning baselines and demonstrating superior computational efficiency. Ensemble strategies further boost performance, with a hard voting ensemble reaching 92.23% accuracy. Compared to resource-intensive large language models such as ChatGPT and GPT-4, the proposed approach delivers competitive results with significantly lower computational overhead, making it suitable for real-time deployment. These findings demonstrate that modern parameter-efficient methods, when combined with modular architectures, offer a scalable and adaptable solution for fake news detection across diverse domains.

Enhancing Web Application Firewalls with BERT-GNN for SQL Injection Detection

Detecting sophisticated SQL Injection (SQLi) attacks remains among the most critical challenges in web applications security. This research study has resulted in an optimised hybrid BERT-GNN pipeline with improved detection accuracy and robustness while reducing false-positive and false-negative rates. SQL queries are tokenised and encoded into contextual BERT embeddings, which then initialise the node features of a Graph Neural Network (GNN) trained to classify each query, with the architecture tuned by Optuna over accuracy, precision, recall, and F1-score. The proposed model achieved 99.67% accuracy, with 99.71% precision, 99.39% recall, and 99.55% F1-score on the attack class. A sensitivity analysis, performed by perturbing graph inputs, further assessed the model robustness and yielded a low mean sensitivity score of 0.0037, indicating stable predictions under such perturbations. The results have demonstrated the potential of a novel hybrid model that couples BERT contextual understanding with the GNN structural modelling to detect sophisticated SQLi attack vectors. For open validation, the dataset, test sets and models are made available at https://github.com/mlily2024/Final-project-SQL-injection-pipeline.