MVXCC-NET: Cross-modal 3D detection of occluded objects based on dual-path information complementation and regional weight modeling

Abstract

Surrounding scene awareness is a core component of self-driving techniques, and the 3D detection accuracy for occluded objects directly determines the system’s scenario adaptability and driving safety. To address the core problem of inter-object occlusions in traffic scenes that lead to reduced 3D detection accuracy and inadequate feature encoding, this paper presents MVXCC-NET, a cross-modal 3D detection network for occluded objects based on dual-path information complementation and regional weight modeling. The presented method adopts an image point cloud fusion framework with a dual branch design. In the image branch, a joint weighting strategy is introduced to progressively enhance image features and strengthen the visible regions of occluded objects. In the 3D point cloud path, a coordinate aware sparse region feature enhancement strategy is introduced to encode point cloud spatial coordinate information explicitly and perform feature weighting, thereby improving the geometric structure perception and spatial modeling ability of LiDAR for sparse point clouds in occluded areas, reducing the loss of geometric features caused by occlusion, and maintaining accurate 3D localization. Meanwhile, through multi-scale feature output and a cross modal deep fusion strategy, the proposed method improves the utilization efficiency of fused features, allowing visual semantic information and spatial geometric information to support each other. Experimental results on the KITTI occluded object dataset show that MVXCC-NET achieves detection accuracies of 55.3%, 72.6%, and 69.1% for pedestrians, cars, and cyclists, respectively. Compared with MVX-Net, POINTPILLARS, POINTRCNN, and SECOND, the mean accuracy is improved by 6.5%, 7.8%, 4.8%, and 14.2%, respectively.