Skip to content
Preprint

SignMatch: Matching Dictionary Signs to Continuous Sign Language Video

Sep 2026 · 0 citations · 50 references
Computer Science

TL;DR

A prototype-structured sign embedding space is learned from continuous video annotated with signs, where each learnable prototype corresponds to a sign class, enabling the matching between dictionary exemplars and continuous sign instances.

Abstract

The objective of this paper is to match dictionary sign videos to corresponding signs in continuous signing videos, where a match is defined by the visual similarity alone - the handshape and motion relative to the body. To achieve this, we learn a prototype-structured sign embedding space from continuous video annotated with signs, where each learnable prototype corresponds to a sign class. Isolated dictionary videos are then mapped into this sign space, enabling the matching between dictionary exemplars and continuous sign instances. This design supports direct dictionary-guided sign matching through embedding similarity and naturally extends to unseen signs using only dictionary exemplars. Experiments on ASL-Citizen dictionary retrieval, ChaLearn OSLWL dictionary-to-continuous sign matching, and using BOBSL's CSLR2 evaluation for automatic sign annotation demonstrate strong generalisation across datasets, tasks and sign languages. Without benchmark-specific supervision, the learned representation transfers effectively across American, British, and Spanish Sign Languages, outperforming prior methods on all three benchmarks. Project page: https://www.robots.ox.ac.uk/~vgg/research/signmatch/

View source

Similar papers

Preprint Sep 2026

SignSeek: Learning Transferable Representations for Sign Dictionary Retrieval

SignSeek sets a new state-of-the-art performance in cross-corpus retrieval on ASL-Citizen, WLASL, and NMFs-CSL without any downstream fine-tuning, surpassing methods explicitly trained on BSL and outperforming prior skeleton-based methods.

Sobhan Asasi, Ozge Mercanoglu Sincan, Richard Bowden · 0 citations
Conference Aug 2026

VSL-HV8: A Vietnamese Sign Language Dataset for Sign Recognition and Translation Research

Vietnamese Sign Language (VSL) remains under-resourced for automatic sign language recognition and translation research. Existing VSL datasets primarily focus on isolated gloss-level recordings, while a large-scale dataset containing continuous sentence-level signing sequences is still unavailable. This limitation rest...

Hien Phuong Nhat Nguyen, Hoai Nhan Nguyen, Thi Diem Tran · 0 citations
Preprint Sep 2026

SignRefine: Adapting Foundational Video Models for Sign Language Generation

SignRefine is proposed, a sign language video generation model that produces comprehensible signing from 2D keypoint conditioning alone, generalizing across appearances and visual conditions, and is preferred by sign language users for visual quality and comprehensibility in more than 80% of comparisons.

A. Pelykh, Edward Fish, Ozge Mercanoglu Sincan et al. · 0 citations

SignTrace: Describe a Sign, Find the Word

Identifying an unfamiliar sign is difficult when a learner remembers its movement but does not know its meaning or formal feature codes. SignTrace addresses this longstanding reverse-lookup problem through natural-language access to a Chinese sign-language dictionary. The system integrates LLM-based dictionary enrichme...

Zeng-Ji Tu, Xing-Ye Zhu, Ning Wang et al. · 0 citations
#computer vision Open access Sep 2026

Zero-Shot Cross-Lingual Recognition of Sign Language Handshapes

This work presents the first zero-shot cross-lingual framework for handshape recognition, transferring from ASL to Catalan Sign Language (LSC), and leverages the decomposition of handshapes into five phonological features shared across both languages, to decode LSC handshapes from predicted features via a composite pho...

Marcel Granero Moya, Carolina Del Corral Farrarós, G. Haro et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.