Shi-Wei Gan

Seeing Semantic Shift: Difference-Aware Sentence-Level Temporal Segmentation of Sign Language Videos

Recent advances in sign language understanding have achieved impressive success on short, single-sentence videos, yet their performance drops sharply when applied to long, continuous sign language videos. To bridge this gap, we focus on a challenging and realistic setting: Visual-only Sentence-level Sign Language Segme...

Bridging the Gap Between Semantics and Reconstruction:Unifying Sign Language Translation and Production

Recent advances in sign language (SL) research have shown a trend toward unifying multiple sign language understanding (SLU) subtasks, such as isolated sign language recognition (ISLR), continuous sign language recognition (CSLR), and sign language translation (SLT), within a single framework, leading to substantial pr...

Sign Language Question Answering: A New Task, Benchmark, and Baseline for Sign Language Understanding

A new task, Sign Language Question Answering (SLQA), which evaluates SL understanding by requiring models to answer arbitrary natural language questions about SL videos, and a simple yet effective baseline model equipped with a Question-Conditioned Modulated Temporal Downsampling module and an in-domain knowledge trans...

SignLlama: Enhancing Gloss-free Sign Language Translation by Prioritizing Visual Features for LLMs

Large Language Models (LLMs) have achieved remarkable success across a wide range of tasks. However, fine-tuning LLMs for Gloss-Free Sign Language Translation (GFSLT) remains a challenge. In this paper, we investigate how to effectively adapt LLMs to the GFSLT task. We show that there are two key issues that need to be...

Learning Effective Sign Features without Text for Gloss-free Sign Language Translation