Skip to content
Open access

FGA: Face Gesture Assistance

Oct 2026 · International Journal of Research Publication and Reviews · 0 citations

Abstract

Traditional desktop interfaces rely on a physical mouse and keyboard, which can be difficult to use for people with limited hand mobility and in hands-busy situations. Face Gesture Assistant (FGA) is an offline desktop interaction prototype that combines facial-landmark pointer movement with speech commands and dictation. The camera pipeline uses OpenCV and MediaPipe Face Landmarker to estimate a facial refer-ence point, map movement to screen coordinates, and smooth pointer motion. Mouth opening is detected using the Euclidean distance between upper- and lower-lip landmarks. The audio pipeline captures 16-kHz mono audio, uses preprocessing and Silero voice activity detection (VAD), and transcribes speech locally with Faster-Whisper. A command handler normalizes text and dispatches supported desktop operations. The camera and speech pipelines run in separate Qt threads within one application process. During detected speech, mouth-click actions are suppressed to reduce unintended clicks caused by speaking-related lip movement. Runtime logs from three Windows laptops show that effective camera rate and end-to-end latency vary by run, while cursor-coordinate processing is comparatively inex-pensive. These findings are engineering observations rather than a controlled accessibility or usability study. The paper describes the implementation, summarizes the available measurements, and identifies additional testing needed to establish reliability.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.