Cross-modal point cloud completion for robust non-contact 3D measurement under occlusion

Abstract

Non-contact 3D measurements acquired by LiDAR, structured-light and depth-camera systems often contain large missing regions under occlusion, limited viewpoints, motion blur and sensor noise. These defects reduce the geometric fidelity of reconstructed shapes and directly affect downstream dimensional analysis, pose estimation and robotic manipulation. This paper presents UBCnet, a cross-modal point cloud completion framework intended to improve the robustness of image-assisted 3D measurement under incomplete observations. The method combines unified feature encoding, bilateral complementary fusion and a consistency-guided reconstruction strategy. First, image and partial-point-cloud features are mapped into a shared latent space to reduce cross-modal inconsistency before fusion. Second, a symmetric bilateral interaction path uses image appearance cues to guide inference of missing 3D regions while using point-cloud geometry to constrain ambiguities introduced by single-view image observations. Third, a coarse-to-fine decoding strategy together with a consistency loss improves both global completeness and local structural accuracy. The framework is evaluated on ShapeNet-ViPC and on a real captured dataset constructed with multi-view RGB images and 3D sensor data. Compared with representative single-modal and multimodal completion methods, UBCnet achieves the best average Chamfer distance (1.167 × 10−3) and F-score (0.859) on ShapeNet-ViPC, with particularly strong gains for thin or concave-convex objects. Controlled robustness sweeps under point-wise noise, missing-point ratio and image degradation further substantiate the claim of robust completion. Real-world experiments show more consistent reconstruction of edges, curved transitions and grasp-relevant structures. The results indicate that the proposed framework can serve as an advanced measurement tool for improving completeness and geometric fidelity in non-contact 3D measurement systems operating under incomplete sampling conditions.