Skip to content

Gesture Control Automation: A Webcam-Based Assistive System for Communication and Computer Control

Aug 2026 · International Journal of Creative and Open Research in Engineering and Management · 0 citations

TL;DR

Gesture Control Automation, a webcam-based system that recognizes custom hand gestures and uses them for communication and computer control and attempts to combine these gesture recognition, communication, and automation systems.

Abstract

Gesture recognition is becoming a useful way of interacting with computers without using traditional devices such as keyboards, mice, and touchscreens. This type of interaction can be especially helpful for people who have physical or communication difficulties. In this paper, we propose Gesture Control Automation, a webcam-based system that recognizes custom hand gestures and uses them for communication and computer control. The system captures live video through a webcam and uses OpenCV for image processing and MediaPipe for hand landmark detection. The detected landmarks are used to identify predefined hand gestures. The main idea of the proposed system is to use the recognized gesture for more than one purpose. A gesture can be converted into text, the text can be converted into speech, and selected gestures can also be used to perform predefined computer operations. To reduce accidental commands, the system can use confidence checking and multiple-frame confirmation before executing an automation action. The proposed approach is simple, low-cost, and does not require special wearable sensors. The system is mainly intended to improve accessibility and provide a common interface for communication and computer control. The work also identifies a gap in existing systems, where gesture recognition, communication, and automation are often developed separately. The proposed system attempts to combine these Keywords— Gesture Recognition; Computer Vision; MediaPipe; OpenCV; Assistive Technology; Automation

View source

Similar papers

Open access Sep 2026

A Vision-Based Human-Computer Interaction System Using Hand Gestures

Human–Computer Interaction (HCI) traditionally depends on physical input devices such as a mouse, keyboard, touchpad, and touchscreen. Although these devices provide efficient interaction, they require direct physical contact and may not always be convenient in situations where touchless interaction is preferred. This...

S. Uma, S. A · 0 citations
Open access Aug 2026

Gesture-Based Computer Interface Using Hand Gestures and Eye Blink Detection

Conventional human-computer interaction relies on physical input devices such as a mouse and keyboard, which is not always convenient, hygienic, or accessible to every user. This paper presents a touchless, webcam-based computer interface that allows a user to control common mouse and navigation functions using hand ge...

Shyalaja L. N., Prajwal P. C., Siddhanth Sasture et al. · 0 citations
Open access 2026

AI Powered Gesture Control for Public Touchscreens

An AI-powered, vision-based gesture control system designed to provide a touchless alternative to conventional public touchscreen interaction using only a standard webcam is developed and demonstrates that a software-only approach using a regular webcam can provide responsive and practical touchless interaction without...

Samyak Kasliwal, Maaz Khan, K. Gaikwad et al. · 0 citations
Conference Aug 2026

Hand Gesture Recognition Using MediaPipe Framework: A Vision-Based Touchless Interaction System

Hand gesture recognition is a key enabling technology in vision-based touchless interaction systems, allowing users to interact with digital applications through natural hand movements without physical contact. Traditional input devices such as keyboards, mouse, and touchscreens present significant limitations in envir...

Avlin Antony, Jiss Kuruvilla, Christina Baiju et al. · 0 citations

Інформаційні технології

A conceptual model of context-adaptive inclusive user interaction with an information system and the external environment is developed and experimentally validate a method of contactless interaction with visual and textual content based on RGB hand tracking and contextual interpretation of gestures.

Удк, О. Я. Барковська · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.