Skip to content
Book Open access

Toward In-the-Wild Gloss-Free Sign Language Translation via Domain-Adaptive Cross-Modal Alignment

Nov 2026 · 0 citations

Abstract

Enabling gloss-free sign language translation (SLT) in unconstrained real-world environments is a critical step toward practical deployment. Existing progress has been driven primarily by benchmarks collected in visually homogeneous laboratory settings, where simplified scenes substantially reduce the difficulty of cross-modal alignment. As a result, a pronounced domain gap emerges when models are evaluated on newly released in-the-wild benchmarks or deployed in real-world environments. Under such a domain shift, genuine sign semantics are easily confounded by nuisance visual factors, leading models to rely on non-semantic scene cues rather than the fine-grained spatio-temporal dynamics of sign articulations. To address this challenge, we propose Domain Adaptive Contrastive Learning (DACL), a domain-adaptive training framework that promotes robust cross-modal semantic grounding in visually unconstrained environments. Specifically, DACL transfers semantic alignment priors from controlled source domains to in-the-wild target domains, while introducing signer-free pure background samples as an explicit regularizer to suppress nuisance-driven correlations and encourage the disentanglement of scene context from sign semantics. Without introducing architecture-specific modifications, DACL achieves a BLEU-4 score of 21.47 (+19.03) on the in-the-wild CE-CSL and 18.60 (+3.68) on OpenASL, substantially outperforming counterparts trained directly on the target domain.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.