MCPF-Net: Multi-Stage LiDAR-Image Collaborative Perception Fusion Network for Point Cloud Semantic Segmentation of Urban Scenes

Abstract

Multimodal fusion unlocks the potential of point cloud semantic segmentation, thereby driving advancements in surface observation and visual perception tasks. Although light detection and ranging (LiDAR) systems capture precise 3D structural geometry and optical images provide rich semantic and textural information, existing fusion methods struggle with limited cross-modal perception and insufficient information complementarity. To address these limitations, we propose a multi-stage LiDAR-image collaborative perception fusion network (MCPFNet) for point cloud semantic segmentation of urban scenes. At the middle fusion stage, the network incorporates an elevation-guided geometric-aware fusion module and a semantic-aware cross-attention fusion module to enable bidirectional feature injection between LiDAR and image modalities. In the late fusion stage, a bidirectional adaptive fusion module further refines semantic representations through gated weighting and bidirectional cross-attention mechanisms. Extensive experiments on three multimodal datasets with different resolutions, i.e., ISPRS Vaihingen, N3C-California, and UAVScenes, demonstrate that MCPFNet outperforms existing fusion methods, achieving mIoUs of 74.51%, 95.15%, and 62.76%, respectively. Hence, our multi-stage fusion and bidirectional interaction strategy is more reliable and accurate than existing methods in performing segmentation across diverse and complex urban scenes.