Optimized Deep Learning Model for Phishing Email Detection Using the Coyote Optimization Algorithm: A Comparative Study of LSTM, BERT, and Random Forest Classifiers

Abstract

Among cybersecurity threats, phishing stands out for how persistently it evolves, using social engineering to pry sensitive data away from users. Blacklists and static rule-based filters, the conventional lines of defence, cannot keep pace with how quickly phishing tactics shift, and even deep learning models, despite their promise, are often held back by poorly tuned hyperparameters and weak generalization to new data. To address this, the present study builds an optimized deep learning framework for email-based phishing detection, pairing the Coyote Optimization Algorithm (COA), a swarm-based metaheuristic modelled on the adaptive hunting behaviour of coyotes, with three classifiers: Long Short-Term Memory (LSTM) networks, Bidirectional Encoder Representations from Transformers (BERT), and Random Forest (RF). Working from a phishing-webpage feature dataset (Tan, 2018; 10,000 labelled instances, 48 features), the models were built through a pipeline of data cleansing, normalization, label encoding, and feature selection, then split 80:20 into training and test sets. COA was used to tune each model's hyperparameters, and results were judged on accuracy, precision, recall, F1-score, and Area Under the Receiver Operating Characteristic Curve (AUC-ROC). The COA-optimized LSTM model came out on top, reaching 96.82% accuracy, 97.72% precision, 97.03% recall, 97.37% F1-score, and 99.61% AUC-ROC, ahead of the COA-optimized BERT (96.26% accuracy, 99.45% AUC-ROC) and Random Forest (94.80% accuracy, 98.89% AUC-ROC) variants. These figures exceed several benchmarks reported in the literature, suggesting that combining sequence-sensitive deep learning architectures with bio-inspired hyperparameter optimization produces a robust, high-precision phishing detection framework fit for deployment across email filtering, enterprise security, and online banking platforms.