ZeroSplat lifts 2D Vision-Language Model priors into 3D space through robust multi-view geometric constraints and enables intrinsic point-level understanding without incurring any additional feature storage, and significantly outperforms state-of-the-art methods across generalized and single-target scenarios while maintaining exceptional efficiency.
Abstract
Recent advancements in 3D Gaussian Splatting (3DGS) have enabled language-guided scene understanding. However, existing Referring 3D Gaussian Splatting (R3DGS) methods are fundamentally restricted to single-target queries. To reflect the ambiguity of real-world instructions, we introduce the Generalized Referring 3D Gaussian Splatting Segmentation (GR3DGS) task, which requires dynamically segmenting an arbitrary number of targets (0, 1, or $N$). To facilitate comprehensive evaluation of this new task, we construct two new benchmarks: GR-LERF and GR-ScanNet. Crucially, existing R3DGS paradigms exhibit fundamental technical bottlenecks that severely limit their performance on the GR3DGS task: they lack intrinsic 3D point-level understanding by operating merely on 2D rendered pixels, and they incur prohibitive computational overhead by requiring per-scene optimization to embed heavy semantic features. To dismantle these bottlenecks, we propose ZeroSplat, a novel training-free and zero-feature framework. ZeroSplat lifts 2D Vision-Language Model (VLM) priors into 3D space through robust multi-view geometric constraints. This strategy enables intrinsic point-level understanding without incurring any additional feature storage. Extensive experiments demonstrate that ZeroSplat significantly outperforms state-of-the-art methods across generalized and single-target scenarios while maintaining exceptional efficiency. Project Page: https://inkmind-ai.github.io/ZeroSplat
3D Gaussian splatting (3DGS) has recently emerged as an efficient and scalable method for high-fidelity 3D scene reconstruction, representation, and real-time rendering. In addition to geometric reconstruction, increasing research attention focuses on enriching 3D Gaussian primitives with semantic information, which can be related to an arbitrary application or domain, as well as common knowledge. However, the existing approaches to semantic 3DGS significantly differ in how semantics are represented, learned, and accessed, which makes systematic analysis difficult. This paper provides a review on semantic extensions to 3DGS. We introduce a unified multi-axis taxonomy that enables us to classify the available methods in terms of five complementary categories: semantic vocabulary space, representation form, functional role, knowledge source, and query mechanism. The analysis reveals key design trade-offs related to the flexibility, efficiency, and semantic expressiveness of the methods. Furthermore, we review datasets, benchmarks, and evaluation metrics used in the field, indicating the diversity of approaches and the lack of common evaluation frameworks. Based on this analysis, we also identify open challenges and possible future research directions. The presented survey is relevant to advances in games and immersive technologies, where semantically enriched real-time 3D representations are essential for interactive environments, AR/VR, and intelligent scene understanding. The systematic analysis presented in this survey aims to facilitate a deeper understanding of semantic 3DGS and support the development of more general, efficient, and task-aware 3D scene understanding systems.
3D Gaussian splatting (3DGS) provides an efficient and expressive scene representation by jointly modeling spatial geometry and appearance, which has led to significant advances in high-fidelity scene reconstruction and novel view synthesis. However, in real-world environments, occlusions from pedestrians or equipment often introduce blurriness, artifacts, and geometric distortions. To address these challenges, this paper proposes a robust 3DGS modeling method enhanced by semantic and geometric consistency. First, the self-supervised foundation model DINOv2 is utilized to extract high-dimensional semantic features, leveraging its superior generalization capabilities to assist in identifying potential dynamic regions. Second, monocular depth estimation and a depth residual mechanism are introduced to construct geometric consistency constraints, enabling the precise localization of areas that violate static assumptions. Finally, a progressive guided probability masking mechanism is designed; it employs an adaptive sigmoid function to achieve a ‘coarse-to-fine’ soft-constraint optimization, effectively mitigating the training instability inherent in traditional binary hard masks. Experimental results on the neural radiance fields (NeRF)-on-the-go, RobustNeRF, and self-collected datasets demonstrate that the proposed method effectively suppresses dynamic artifacts and improves reconstruction quality. The proposed approach achieves competitive or superior performance compared with 3DGS, SpotLessSplats, T-3DGS, and RobustSplat on standard image-quality metrics, including peak signal-to-noise ratio, structural similarity index measure, and learned perceptual image patch similarity.
Wen Zheng, Guo Bao, Wenda Wang et al.· Engineering Research Express· 0 citations
Recent generalizable 3D Gaussian Splatting models have advanced long-sequence novel view synthesis (NVS), but at the cost of substantial redundant computation. We identify that the redundancy can be mitigated based on two observations: (i) high-precision geometry is not strictly required for high-quality NVS; (ii) appearance learning is generally easier than geometry recovery. Motivated by these insights, we propose an asymmetric architecture that decouples geometry and appearance modeling. The geometry branch processes coarse-grained tokens with most of the parameters for multi-view reconstruction, while the appearance branch operates on fine-grained tokens to capture details using significantly fewer parameters. The two branches interact through bilateral connections, enabling mutual guidance for their respective tasks. This task-aware asymmetry reduces the computational redundancy and allocates the computation more judiciously, thereby increasing parameter efficiency and enabling smaller models to achieve strong performance. On 32-view 960P inputs, our model matches optimization-based methods while delivering nearly 800x speedup, and surpasses the zero-shot performance of state-of-the-art generalizable models with markedly fewer parameters and reduced training/inference overhead, achieving an overall efficiency improvement.
Yingji Zhong, Dave Zhenyu Chen, Fu-Zhao Ou et al.· 0 citations
OutLangSplat is presented which adapts language Gaussian representations to UAV outdoor scenes by improving feature representation and aggregation reliability, and is the first accessible dataset of open-vocabulary 3D scene understanding for UAV outdoor scenes.
Xiaosheng Yan, Hefeng Wu, Yanghui Xu et al.· 0 citations
Real-time visualization of massive 3D virtual environments, such as city-scale digital twins and immersive simulations, remains a core challenge in computer graphics. While 3D Gaussian Splatting (3DGS) achieves photorealistic rendering at interactive rates, its explicit Gaussian representation incurs substantial GPU memory overhead, limiting deployment on resource-constrained devices. Existing 3DGS Level-of-Detail (LoD) methods are often data-dependent, requiring access to large collections of original training images, and they rarely adapt to dynamic hardware budgets, leading to unstable performance on low-VRAM GPUs. We present InfoLoD, a data-free and hardware-aware framework for hierarchical LoD construction and large-scale 3DGS rendering. InfoLoD introduces a Fisher-guided self-distillation scheme that uses the Fisher Information Matrix (FIM) to select geometrically valid, information-rich pseudo viewpoints, enabling LoD training directly from a pre-trained 3DGS model without any original images. To operate under strict VRAM constraints, we further propose a budget-aware out-of-core streaming system with spatio-temporal working-set management and resource-adaptive LoD scheduling to balance fidelity and throughput. Experiments on challenging outdoor scenes show that InfoLoD enables stable real-time rendering on resource-constrained GPUs, delivering higher efficiency than prior baselines while maintaining competitive visual quality.
Zhenyu Xia, Pengcheng Han, Lin Chen et al.· IEEE Transactions on Visuali...· 0 citations
Semantic segmentation in 3D Gaussian Splatting (3DGS) is crucial for advancing 3D scene understanding. Existing methods predominantly rely on feature distillation, which incurs substantial per-scene training overhead and often yields blurred segmentation boundaries. We identify that these boundary artifacts are driven in part by insufficient viewpoint coverage and boundary overflow of anisotropic Gaussian primitives. To address these challenges, we propose VCAR, a training-free coarse-to-fine segmentation strategy based on View Completeness and Axis-aware Boundary Refinement. In the coarse stage, a visibility-based weighted multi-view voting scheme rapidly localizes the target. In the fine stage, an object-centric sphere derived from the coarse result generates supplementary viewpoints via Spherical Spiral Sampling (SSS), allowing multi-view voting on the augmented views to precisely refine object boundaries and suppress irrelevant 3D Gaussians. Moreover, we introduce Axis-aware Boundary Refinement (ABR) to mitigate artifacts from anisotropic primitives. By decomposing the projected 2D covariance into per-axis contributions, ABR identifies the dominant axis responsible for boundary leakage and applies targeted anisotropic compression exclusively along that axis. Extensive experiments on NVOS and LERF demonstrate that VCAR achieves state-of-the-art segmentation accuracy and efficiency without training. Our code is available at https://github.com/DDKK0526/VCAR.
Kun-Chun Cao, Di Wang, Haiming Zhu et al.· 0 citations