Skip to content
#edge computing Open access

SIGN-AIR: real-time sign language recognition and translation using gesture-aware neural networks on aerial robotic platforms

Aug 2026 · Scientific Reports
Hand Gesture Recognition Systems

Abstract

Abstract Communication barriers experienced by deaf and hard-of-hearing individuals remain a significant challenge, particularly in situations where conventional communication infrastructure is unavailable or fixed-camera systems have limited coverage. This paper presents SIGN-AIR, a UAV-assisted framework for real-time sign language recognition and bidirectional translation. Unlike conventional approaches based on fixed cameras, wearable sensors, or depth cameras, the proposed framework integrates a drone-mounted stabilised RGB camera with onboard edge computing to enable flexible gesture recognition in indoor and outdoor environments. The system combines MediaPipe-based hand landmark extraction with a hybrid CNN–BiLSTM architecture to model both the spatial and temporal characteristics of American Sign Language gestures. Experimental evaluation on a combined dataset comprising the ASL Alphabet and WLASL achieved an overall recognition accuracy of 98.63%, together with high precision, high recall, and a median inference latency of less than 2 s. These results demonstrate the technical feasibility of the proposed UAV-assisted recognition framework under the evaluated experimental conditions and indicate its potential to support communication in scenarios where fixed-camera systems may be affected by occlusions, limited fields of view, or environmental variability. Furthermore, the proposed bidirectional translation pipeline has the potential to facilitate both sign-to-speech and speech-to-sign communication for prospective applications in accessibility, education, public service interactions, and emergency response. Although the current study demonstrates promising technical performance, comprehensive real-world field validation under representative operational conditions remains an important direction for future work.

Read PDF

Similar papers

#computer vision Review Sep 2017

Agile Software Development Methods: Review and Analysis

Agile - denoting "the quality of being agile, readiness for motion, nimbleness, activity, dexterity in motion" - software development methods are attempting to offer an answer to the eager business community asking for lighter weight along with faster and nimbler software development processes. This is especially the case with the rapidly growing and volatile Internet software industry as well as for the emerging mobile application environment. The new agile methods have evoked substantial amount of literature and debates. However, academic research on the subject is still scarce, as most of existing publications are written by practitioners or consultants. The aim of this publication is to begin filling this gap by systematically reviewing the existing literature on agile software development methodologies. This publication has three purposes. First, it proposes a definition and a classification of agile software development approaches. Second, it analyses ten software development methods that can be characterized as being "agile" against the defined criterion. Third, it compares these methods and highlights their similarities and differences. Based on this analysis, future research needs are identified and discussed.

P. Abrahamsson, O. Salo, Jussi Ronkainen et al. · 728 citations · ⚡54
#machine learning Review Open access Oct 2014

Software development in startup companies: A systematic mapping study

Context: Software startups are newly created companies with no operating history and fast in producing cutting-edge technologies. These companies develop software under highly uncertain conditions, tackling fast-growing markets under severe lack of resources. Therefore, software startups present a unique combination of characteristics which pose several challenges to software development activities. Objective: This study aims to structure and analyze the literature on software development in startup companies, determining thereby the potential for technology transfer and identifying software development work practices reported by practitioners and researchers. Method: We conducted a systematic mapping study, developing a classification schema, ranking the selected primary studies according their rigor and relevance, and analyzing reported software development work practices in startups. Results: A total of 43 primary studies were identified and mapped, synthesizing the available evidence on software development in startups. Only 16 studies are entirely dedicated to software development in startups, of which 10 result in a weak contribution (advice and implications (6); lesson learned (3); tool (1)). Nineteen studies focus on managerial and organizational factors. Moreover, only 9 studies exhibit high scientific rigor and relevance. From the reviewed primary studies, 213 software engineering work practices were extracted, categorized and analyzed. Conclusion: This mapping study provides the first systematic exploration of the state-of-art on software startup research. The existing body of knowledge is limited to a few high quality studies. Furthermore, the results indicate that software engineering work practices are chosen opportunistically, adapted and configured to provide value under the constrains imposed by the startup context.

Nicolò Paternoster, Carmine Giardino, M. Unterkalmsteiner et al. · 394 citations · ⚡54

Related blog posts

Microsoft Research Blog Aug 31, 2026

GigaPath-Flash and GigaTIME-Flash: Toward population-scale discovery with efficient pathology foundation models

What if pathology foundation models could do more with less? GigaPath-Flash and GigaTIME-Flash cut computational demands while maintaining strong performance, opening the door to larger studies and broader exploration. The post GigaPath-Flash and GigaTIME-Flash: Toward population-scale discovery with efficient pathology foundation models appeared first on Microsoft Research.

MIT News · Artificial Intelligence Aug 27, 2026

Looking beyond natural sequences

A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.