Sailfish optimized MobileNet for robust recognition of American sign language alphabets and digits

Abstract

American sign language (ASL) is widely used for communication among the deaf and hard-of-hearing communities. This study aims to develop an optimized deep learning (DL) model for recognizing 36 static ASL gestures representing English alphabets (A–Z) and digits (0–9). A publicly available ASL dataset was used, and five convolutional neural network (CNN) architectures—AlexNet, GoogleNet, Inception V3, MobileNet, and ResNet50—were implemented as baselines. MobileNet was further optimized using the sailfish optimization (SFO) algorithm to fine-tune key hyperparameters and architectural settings. The models were evaluated using accuracy, macro precision, recall, F1-score, specificity, Cohen’s Kappa, Matthews correlation coefficient (MCC), balanced accuracy, Jaccard index, and error rate. The SFO-enhanced MobileNet achieved the highest performance, with 98.28% accuracy, 98.27% macro F1-score, 99.95% specificity, and a 1.72% error rate, outperforming all baselines across metrics. These results demonstrate that SFO optimization significantly improves MobileNet’s ability to classify ASL gestures accurately and efficiently. The proposed model’s high accuracy, robustness, and low inference time (22 ms) make it suitable for real-time sign language interpretation tools and assistive communication devices, supporting broader accessibility and inclusivity.