Supercell thunderstorm paper publishing-based optimization enabled deep learning for sign language recognition using videos

Abstract

Sign language is an essential communication system for hearing-impaired individuals, which mainly depends upon complex hand gestures and facial expressions. Automating Sign Language Recognition (SLR) from videos can enhance barrier-free communication, yet it remains challenging due to the subtle nature of signs in diverse environments. Existing modules often need extensive manual feature extraction and struggle with real-time applications due to high latency. Thus, an effective model for SLR using videos is presented, named Supercell Thunderstorm Paper Publishing-based Optimization enabled Convolutional Grid Long Short-Term Memory (STPPO_CGLSTM). The frames are extracted from the input video. Then, the Arithmetic Mean Filter is employed for pre-processing the extracted frames. Moreover, humans are segmented by employing correlational spectral clustering. Thereafter, hand action unit detection is carried out by considering the AU-Net. Following this detection task, effectual features are extracted. Lastly, sign language is recognized by utilizing CGLSTM, where the hyperparameters of CGLSTM are trained using STPPO. The analytical measures, namely, accuracy, Positive Predictive Value (PPV), Negative Predictive Value (NPV), and False Omission Rate (FOR), obtained better outcomes for STPPO_CGLSTM, which is 96.707%, 98.219%, 94.016% and 5.984% using k-fold cross validation.