Jul 2026· The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences· 0 citations· 9 references
TL;DR
A SAM-guided framework for point cloud oversegmentation that significantly improves boundary recall and maintains high oracle accuracy while maintaining high oracle accuracy, and generalizes well to unseen datasets without retraining, showing strong cross-dataset inference capability.
Abstract
Abstract. Large-scale LiDAR point clouds provide rich geometric information, yet learning effective structural representations remains challenging due to the misalignment between semantic categories and geometric structures. To address this issue, we propose a SAM-guided framework for point cloud oversegmentation. We transfer grouping knowledge from 2D vision by constructing a large-scale oversegmentation dataset using the Segment Anything Model (SAM) on bird’s-eye-view projections. Based on these grouping priors, a structure-aware point cloud encoder is learned via a distillation objective that enforces intra-region compactness and inter-region separation in the embedding space. The proposed approach does not rely on semantic supervision and directly learns generalizable structural representations. Experiments on various benchmark datasets (STPLS3D, Toronto-3D, DALES, and S3DIS) demonstrate that the proposed method achieves competitive performance. In particular, it significantly improves boundary recall (e.g., 92.21% on STPLS3D and 93.47% on Toronto-3D) while maintaining high oracle accuracy (up to 97.62%). Moreover, the model generalizes well to unseen datasets without retraining, showing strong cross-dataset inference capability.
A lightweight boundary-aware learning framework that explicitly models boundary regions during training is proposed, showing that incorporating boundary-aware supervision provides an effective and efficient approach to improving segmentation quality in challenging regions.
Waseem Iqbal, J. Paffenholz· The International Archives o...· 0 citations
Learning-based global point cloud registration has achieved remarkable progress, yet its reliance on geometric representations makes existing methods sensitive to variations in point density, scan pattern, viewpoint, and sensor characteristics. We propose CVSD-Reg, a robust global LiDAR registration framework that distills visual semantic priors from a vision foundation model into LiDAR representations. In Stage 1, a Point Transformer V3 student learns from a frozen DINOv2 teacher through contrastive distillation and spherical-manifold alignment, which preserves the hyperspherical geometry of the teacher embedding space. Self-supervised InfoNCE consistency and soft $\mathrm{SE}(3)$ invariance further encourage viewpoint-robust descriptors. In Stage 2, the distilled representation is adapted to registration through correspondence learning, density-aware point-dropout augmentation, and end-to-end pose optimization. With a single checkpoint, CVSD-Reg generalizes to both single-sensor and zero-shot cross-sensor scenarios without sensor-specific adaptation and remains entirely camera-free at inference. On KITTI, nuScenes, and HeLiPR, CVSD-Reg achieves strict success rate (SR@0.5\,m/$1^\circ$) of 97.7$\%$, 99.0$\%$, and 99.3$\%$, respectively, including 97.3$\%$ on sparse 16-beam Velodyne scans. It outperforms state-of-the-art geometric registration methods by up to 44.0 percentage points without requiring camera inputs or post-hoc ICP refinement.
Eunsoo Im, Junghun Suh, Gyeonggwan Lee et al.· 0 citations
Recently, open-vocabulary zero-shot 3D scene understanding using vision foundation models has emerged as a promising alternative to data-intensive supervised methods. However, deploying these models in real-world scenarios is severely hindered by their inability to efficiently handle streaming RGB-D inputs and their inherent vulnerability to noise 2D segmentation masks. To address these critical limitations, we propose Stream3Dv2, a novel training-free framework designed for robust streaming 3D perception. Stream3Dv2 processes sequential data through an original nested local-to-historical architecture, capturing multi-view consistency while circumventing the high computational overhead so as to support timely responses. At its core, we introduce a comprehensive geometric-semantic fusion mechanism that resolves geometric noise and semantic ambiguity by explicitly utilizing semantic guidance and formulating 3D segmentation as solving point-and-set merging and partitioning problems. Furthermore, we present an innovative manifold-distance-based point cloud refinement strategy. This approach leverages local manifold graphs for point-to-manifold optimization that mitigates the boundary delineation failures caused by Euclidean-distance metrics, and employs geometric bounding boxes to dynamically activate and update historical instances for achieving rapid manifold-to-manifold refinement. Extensive experiments on public datasets demonstrate that Stream3Dv2 consistently outperforms existing baselines in foundational open-vocabulary streaming 3D segmentation and detection. Finally, we show that integrating our framework with an LLM-based agent enables advanced language-driven 3D scene understanding, underscoring its potential for open-world embodied intelligence. Code will be updated at https://github.com/SubmissionsIn/Stream3D.