Intelligent 3D object recognition in complex scenes based on deep RGB–LiDAR fusion

Abstract

Three-dimensional object recognition is an important perception task in autonomous driving, intelligent transportation, and mobile robotics, where models must identify object categories while estimating spatial location, size, and orientation. RGB cameras provide dense semantic and texture information, whereas LiDAR sensors provide accurate depth and geometric structure. However, camera-based methods are sensitive to illumination and depth ambiguity, while LiDAR- based methods may degrade under sparse point distribution, long-distance observation, and occlusion. Existing RGB– LiDAR fusion methods often rely on feature concatenation or fixed fusion weights, which insufficiently consider scene- dependent modal reliability. To address this issue, this paper proposes a reliability-gated cross-modal fusion model, RG- CMF, for complex-scene 3D object recognition. The method integrates calibration-aware RGB–LiDAR alignment, dual- branch feature extraction, reliability-gated fusion, and geometry–semantic consistency optimization. Experiments on the KITTI validation set show that RG-CMF achieves 84.7 ± 0.7, 57.1 ± 0.9, and 68.5 ± 0.9 moderate-level AP for cars, pedestrians, and cyclists, respectively. Compared with PV-RCNN, the mean translation error decreases from 0.351 ± 0.012 m to 0.319 ± 0.010 m. Ablation results further indicate that reliability-gated fusion contributes the largest performance gain, while consistency loss improves localization accuracy. The findings suggest that explicit reliability modeling can improve the accuracy, robustness, and interpretability of RGB–LiDAR 3D object recognition under tested complex-scene conditions.