Vision-Based Gesture-Controlled Mobile Robot for Human–Robot Interaction Using ROS 2

Abstract

Abstract This paper presents the design and implementation of a gesture-controlled differential-drive mobile robot for human–robot interaction (HRI), using the Robot Operating System(ROS 2) framework. The proposed system introduces a novel, vision-based gesture recognition approach using a Raspberry Pi and an onboard camera, combined with a custom spatial AI algorithm for real-time interpretation of human hand gestures. Unlike conventional systems that rely on physical interfaces, the robot is capable of recognizing intuitive gestures—such as start, stop, and directional commands—to control its movement without any contact-based input. The gesture recognition module integrates lightweight machine learning models optimized for embedded deployment, ensuring accurate and low-latency classification of hand signals in dynamic environments. ROS 2 serves as the middleware for seamless integration of sensory data and control of the differential-drive mechanics, enabling the robot to perform core mobility tasks such as forward, backward, turning, and halting. Experimental evaluations in real-world settings demonstrate the system’s high responsiveness, robustness, and adaptability, underscoring its potential for natural, non-invasive interaction in service robotics, assistive applications, and collaborative human–robot environments.