Multi-angle 3D reconstruction of a scene with adaptive filtering for robotic grasping on the Jetson Nano platform
TL;DR
The process of creating an integrated system for high-precision autonomous object grasping by an ABB IRB 140 industrial robot based on RGB-D perception and deep learning methods is presented and the practical applicability of the system for autonomous manipulation in production environments with partial occlusion and variable scene geometry is demonstrated.
Abstract
The process of creating an integrated system for high-precision autonomous object grasping by an ABB IRB 140 industrial robot based on RGB-D perception and deep learning methods is presented. A fully functional real-time system for the resource-limited NVIDIA Jetson Nano platform is proposed, combining multi-angle 3D reconstruction of the scene with the exclusion of the central zone of the increased error, adaptive processing of depth data and neural network object detection based on YOLOv3. An algorithm for merging point clouds with weighted averaging in voxel representation and adaptive filtering by manipulator configuration has been developed; an ablation study of the contribution of the central zone mask, multi-angle averaging and adaptive filtering to the final capture accuracy has been conducted. The system achieves a gripping accuracy of 97.96 % with an average cycle time of 4.2 ± 0.8 s for orderly stacking and 94.74 % at 6.8 ± 1.5 s for disordered bales with an average localization error of 2.3 ± 1.5 cm. The reliability of the results is confirmed by statistical analysis and comparison with the methods of 6IMPOSE, PVN3D+ and GG-CNN. The advantage of the proposed approach in terms of capture accuracy and occlusion resistance at a comparable processing time is shown. The results obtained demonstrate the practical applicability of the system for autonomous manipulation in production environments with partial occlusion and variable scene geometry.