Gesture Recognition through Object Detection for Efficient Human-Robot Collaboration

Abstract

With the ever-changing and innovative manufacturing and industrial revolution, there has been a major focus on harnessing technology to improve productivity, preserve the environment, and tackle socio-economic issues. Manufacturing processes with Human Robot Collaboration (HRC) systems and gestural interaction are becoming more advanced for better safety, and usability is also becoming a critical point within these processes. This study proposes the integration of a high-level object detection model into the Digital Twin space. The object detection model determines and pinpoints the position of humans and various objects within the same realm, which is very important for the effective and safe interaction between the human and the computer. Additionally, for the purpose of improving the model's accuracy, a technique of transfer learning was adopted, which employs the vast obtainable datasets that came from multiple use cases. The development and application of this Digital Twin allow various conditions to be created for the simulation of the environment in which the robot will operate. With the introduction of 3D space modeling, the Digital Twin can provide simulation of various environmental factors during training. By integrating the Digital Twin with the physical robot system, this simulation framework makes it easy to conduct realistic and precise robot operation and gesture control simulations. The integration of these methods guarantee the efficacy and flexibility of the system, thus enhancing the use and application of Human Robot Collaboration (HRC) and gestural interaction.