A Greedy Skeleton Retrieval Framework for Vietnamese Text-to-Sign Generation

Abstract

Sign Language Production (SLP) plays a crucial role in bridging the communication gap between the Deaf community and broader society, functioning alongside Sign Language Translation (SLT) and Recognition (SLR). In addition to the limited scale of available data, research on Vietnamese Sign Language (VSL) is further hindered by the absence of standardized gloss annotations. Glossing serves as the syntactic representation of sign language; it is an essential intermediary because spoken and signed languages possess fundamentally different grammatical structures, cognitive logic, and word orders. To address these challenges, we propose a lightweight and effective framework for low-resource text-to-sign generation. Specifically, the proposed pipeline first employs a pretrained Vietnamese Transformer encoder-decoder model to translate spoken sentences into intermediate gloss sequences to bridge this linguistic gap. Subsequently, leveraging a data-driven dictionary that maps glosses to skeletal representations, we introduce an efficient greedy-based retrieval strategy to extract and match the corresponding skeletal segments. Finally, these retrieved pose sequences are seamlessly synthesized into continuous sign language videos. Experimental results demonstrate that the proposed framework is effective in a low-resource setting, achieving strong performance in text-to-gloss translation and improved coverage in gloss-to-skeleton mapping compared to conventional tokenization-based approaches. In addition, the interpolation strategy significantly enhances motion smoothness during rendering. These findings suggest that the proposed method provides a practical baseline for text-to-sign generation in under-resourced sign languages.