Advances in Indian Sign Language Processing: A State-of-the-Art Review and a Real-Time Bidirectional Translation Architecture

Abstract

This paper presents a comprehensive review of existing datasets, methodologies, and experimental frameworks for Indian Sign Language (ISL) recognition and translation, highlighting recent advancements driven by deep learning architectures such as Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, and Transformers. Despite this progress, the review finds that current systems remain limited by dataset diversity, lack of standardization, and insufficient support for real-time, user-centric applications. Building on these findings, this study proposes an integrated ISL framework, “Beyond,” that combines dataset curation with real-time, bidirectional, and multilingual (Hindi-English) translation, supported by a finger-spelling fallback for out-of-vocabulary words. The framework further incorporates word-level video alignment, background-normalized gesture recognition, predictive text support, and accessibility-oriented controls, along with a proposed multimodal dataset designed for linguistic alignment and temporal coherence. Together, these contributions aim to bridge ISL research and real-world deployment, supporting scalable, real-time, and inclusive communication for the Deaf and Hard-of-Hearing community.