Toward Deployable Humanoid Mobile Manipulation in Industrial Manufacturing: System Integration and Part-Sorting Validation

Abstract

In this paper, we present a mobile manipulation system for humanoid robots to solve industrial part-sorting tasks, in which the robot needs to pick up a part from one workstation and transport it to another. Several functional modules are included in the system. The lower-body control module comprises locomotion and navigation, where the MPC-WBC framework is designed to maintain dynamic balance during walking, while a navigation model simultaneously localizes the real-time pose of the robot and outputs the desired walking control signals to ensure that the humanoid robot moves accurately between different workstations. In terms of humanoid upper-body manipulation, we first propose a visual perception model capable of segmenting, selecting, and localizing the target part from a set of candidates in the environment; an arm trajectory planning model is then employed to execute the manipulation. Our system incorporates different types of visual perception (learning-based and Augmented Reality (AR)-tag-based) and arm trajectory planning (closed-loop learning-based and model-based) methods to enable our system to work in a wider range of industrial part-sorting settings. Our system is evaluated on two practical industrial part-sorting tasks, e.g. Set Parts Supply (SPS) and Surface Mount Technology (SMT), demonstrating satisfactory performance, flexibility, and generalizability, with strong potential for deployment in real industrial settings. More information and our code can be found on the project website: https://humanoid-mobile-manipulation.github.io/ Note to Practitioners—The techniques employed in our system are simple and well-established, making them transferable to other humanoid robot platforms. The qualitative and quantitative analyses of our system and each functional module on two industrial part-sorting tasks can provide practitioners with useful insights into system design and functional module selection, helping to avoid potential pitfalls. Nevertheless, practical deployment requires substantial engineering effort, as humanoid robot system development is complex and typically demands collaborative work from a team with diverse expertise, including the design and tuning of MPC, WBC, Simultaneous localization and mapping (SLAM), and learning-based models. Empirically, the most demanding and decisive functional module in the full-cycle part-sorting task is manipulation, since the process of establishing stable physical contact with the object is usually unpredictable. When task variation is relatively low, model-based manipulation methods are preferable as the primary approach, given their lower development costs and higher accuracy. In contrast, when task variation is high, learning-based methods are recommended for their greater robustness.