Static and Dynamic MSL Gesture Recognition with Video Data Using LSTM

Abstract

Communication is fundamental to daily life, yet individuals with speech disabilities often face challenges in interacting with others due to language barriers. Malayalam Sign Language (MSL), introduced by the National Institute of Speech and Hearing [11] (NISH), is a relatively new regional sign language that requires effective technological support for recognition and learning. This study presents a deep learning-based MSL recognition system using a newly developed dataset containing more than 30 videos for each sign. The proposed approach employs computer vision techniques for landmark extraction and a Long Short-Term Memory (LSTM) network to recognize both static and dynamic MSL gestures. The model achieved a training accuracy of 94.09% and a testing accuracy of 93.95%, demonstrating reliable recognition performance. To enhance accessibility and learning, a user-friendly web application was also developed with interactive learning modules and live sign recognition features, enabling users to practice MSL independently. This work is designed to facilitate effective communication, improve learning experiences, and encourage social integration for the deaf and hard-of-hearing community in Kerala. It also provides a framework for further advancements in Malayalam Sign Language recognition and intelligent assistive systems.