A Dual-Arm Robotic System for Autonomous Connector Assembly via Vision-Tactile Sensing and Contact-Based Pose Refinement

Abstract

The automation of Deformable Linear Object (DLO) manipulation remains a key challenge in industrial production, where tasks such as connector assembly are still largely performed manually. Although prior work has demonstrated reliable wire terminal pose estimation and insertion monitoring using vision and tactile sensing, these approaches typically assume a fixed and known connector pose, limiting their applicability in flexible manufacturing scenarios. This paper presents a dual-arm robotic system for fully autonomous connector assembly. Two UR5e manipulators are employed: one dedicated to wire perception and insertion, and the other to connector localization and manipulation. To address the limitations of vision-only pose estimation, a multi-stage connector pose estimation strategy is proposed. First, a coarse 6D pose is obtained using a deep learning model trained on synthetic CAD data. Subsequently, the pose is refined through force-controlled contact scanning leveraging a wrist-mounted six-axis force/torque sensor. The acquired contact points are registered to the nominal CAD model via Iterative Closest Point, resulting in sub-millimeter localization accuracy. This connector pose strategy is seamlessly integrated with stereo-based wire perception and tactile-guided insertion monitoring, including pull-test validation. Experimental results demonstrate that the proposed system effectively compensates for connector pose uncertainty, achieving a 93.3% connector-level success rate and confirming its suitability for autonomous wire harness manufacturing.