Intelligent Touchless Human-Computer Interaction Using Gesture Recognition and Voice Commands
TL;DR
A real-time AI-powered gesture recognition and voice-controlled automation system that enables users to interact with computers without traditional input devices such as keyboards and mice and has potential applications in smart homes, healthcare systems, educational platforms, virtual reality environments, and assistive technologies.
Abstract
The growing demand for touchless and intelligent computing systems has increased the importance of advanced Human-Computer Interaction (HCI) technologies. This paper presents a real-time AI-powered gesture recognition and voice-controlled automation system that enables users to interact with computers without traditional input devices such as keyboards and mice. The proposed system utilizes MediaPipe-based hand landmark detection, OpenCV image processing, and machine learning techniques to recognize hand gestures and convert them into computer control actions. In addition, a voice assistant module is integrated to execute voice commands for system operations and application control.The developed system supports various functionalities including cursor movement, click operations, scrolling, volume control, screenshot capture, and voice-based task execution. Experimental evaluation was conducted under different operating conditions to assess recognition accuracy, response time, and system reliability. The results demonstrate efficient real-time performance, accurate gesture classification, and smooth user interaction with minimal latency. The integration of gesture recognition and voice assistance provides a user-friendly, cost-effective, and accessible solution for touchless computing environments. The proposed approach has potential applications in smart homes, healthcare systems, educational platforms, virtual reality environments, and assistive technologies.