Multimodal fusion unlocks the potential of point cloud semantic segmentation, thereby driving advancements in surface observation and visual perception tasks. Although light detection and ranging (LiDAR) systems capture precise 3D structural geometry and optical images provide rich semantic and textural information, existing fusion methods struggle with limited cross-modal perception and insufficient information complementarity. To address these limitations, we propose a multi-stage LiDAR-image collaborative perception fusion network (MCPFNet) for point cloud semantic segmentation of urban scenes. At the middle fusion stage, the network incorporates an elevation-guided geometric-aware fusion module and a semantic-aware cross-attention fusion module to enable bidirectional feature injection between LiDAR and image modalities. In the late fusion stage, a bidirectional adaptive fusion module further refines semantic representations through gated weighting and bidirectional cross-attention mechanisms. Extensive experiments on three multimodal datasets with different resolutions, i.e., ISPRS Vaihingen, N3C-California, and UAVScenes, demonstrate that MCPFNet outperforms existing fusion methods, achieving mIoUs of 74.51%, 95.15%, and 62.76%, respectively. Hence, our multi-stage fusion and bidirectional interaction strategy is more reliable and accurate than existing methods in performing segmentation across diverse and complex urban scenes.
Huchen Li, Wu-da Huang, Xiangda Lei et al.· Remote Sensing· 1 citation
3D Gaussian Splatting enables efficient novel view synthesis, but accurate mesh reconstruction remains difficult in weakly observed and occluded regions, where Gaussian primitives may grow into unstable or geometrically inconsistent structures. We propose CoMVS-GS, a general surface reconstruction framework that combines Multi-View Stereo with Gaussian splatting. CoMVS-GS initializes Gaussian primitives from dense multi-view stereo points with pre-flattened scales and normal-aligned orientations, providing stronger geometric priors than sparse structure-from-motion initialization and reducing ambiguity during early optimization. It further introduces PatchMatch-3DGS Mutual Supervision, where Gaussian-rendered depths and normals initialize PatchMatch refinement, and refined PatchMatch depths supervise Gaussian optimization to improve weakly constrained geometry. For surface extraction, CoMVS-GS replaces truncated signed distance field voxel fusion with a Delaunay graph-cut meshing pipeline, reducing sensitivity to voxel resolution while preserving visibility-consistent surface evidence. Experiments on DTU, GauU-Scene V2, and MatrixCity show that CoMVS-GS remains competitive on object-level reconstruction and improves geometric accuracy and mesh compactness in outdoor scenes while maintaining high rendering quality.
Shihan Chen, Junjing Zhang, Qingsong Yan et al.· 0 citations
Abstract. The proliferation of continuous Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS) has shifted the paradigm of 3D aerial reconstruction from relying solely on geometric stereo matching to inverse rendering optimization. However, while these emerging rendering-based frameworks excel in synthesizing photo-realistic novel views, their capability to extract accurate surfaces in complex aerial scenarios remains ambiguous compared to traditional methods. To establish a clearer understanding, this study presents a comprehensive evaluation of five representative frameworks spanning traditional Structure from Motion (SfM), purely Signed Distance Field (SDF) representations, unstructured 3D Gaussians, hybrid voxel-Gaussians, and strictly explicit sparse voxels. By systematically standardizing identical computational environments, inputs, and unified mesh-extraction pipelines on both real-world airborne LiDAR datasets and synthetic cityscapes, we assess their performance regarding visual fidelity, geometric accuracy, and resource efficiency. The experimental results reveal that while traditional MVS produces the highest overall geometric precision by strictly enforcing multi-view rigid geometry, it is prone to failures in texture-less regions. Among rendering-based representations, a fundamental trade-off exists: highly flexible, unstructured 3DGS achieve highest visual scores but degrade the underlying geometric surfaces; conversely, explicitly structured techniques demonstrate distinct superiority in regularizing topological coherence and floating artifact suppression. Furthermore, we observe that integrating structured voxels avoids the severe memory bottlenecks associated with extracting geometries from chaotic unorganized primitives. These empirical findings emphasize that for large-scale aerial photogrammetry, integrating explicit spatial structuralization into differentiable rendering pipelines is imperative for achieving scalable operations and bridging the geometric accuracy gap with traditional methods.
Shihan Chen, Zhaojin Li, Qingsong Yan et al.· The International Archives o...· 0 citations