This work investigates the integration of geometric priors, in the form of predicted normal and depth maps, into the 3DGS framework to improve the reconstruction quality and reveals that multi-view predictions, as they are done by the recent visual geometry grounded transformer (VGGT), outperform single-view alternatives.
Abstract
3D Gaussian splatting (3DGS) has emerged as a widely-used tool for novel view synthesis, offering real-time rendering in a sparse representation. However, the method's reliance on structure-from-motion initialization and photometric optimization can lead to suboptimal geometric reconstruction, particularly for objects with high specularity. In this work, we investigate the integration of geometric priors, in the form of predicted normal and depth maps, into the 3DGS framework to improve the reconstruction quality. We analyze the effect of incorporating these priors into GS-based methods and our evaluation reveals that multi-view predictions, as they are done by the recent visual geometry grounded transformer (VGGT), outperform single-view alternatives. A major factor is the existence of a confidence map for the estimations, which comes as a by-product of multi-view models and which can significantly improve the effectiveness of priors by weighting each prediction appropriately. Extensive experiments on standard benchmarks show consistent improvement in reconstruction quality and significant gains in complex scenes including specular objects.
Despite recent advancements in single-view 3D re construction leveraging multi-view diffusion (MVD), they still suffer from multi-view inconsistency and low-fidelity reconstruction. Existing solutions often introduce priors through prolonged, large-scale training, which is costly, and they struggle with aligning features with conditioning signals. In this work, we propose GeometryAP, a training-free framework for consistent 3D content generation via geometry-informed auxiliary priors in a two-stage "generate-then-reconstruct" pipeline. Our approach consists of two critical components: (1) To enhance cross-view consistency, we design the Auxiliary Prior-guided Attention Mechanism (APAM), which uses the MVD's own pseudo-normal noise features as an auxiliary prior. APAM fuses this 3D prior information to activate crucial attention regions, ensuring the 3D consistency of generated multi-view images. (2) To achieve robust reconstruction from the generated multi view images, we propose the 3D Gaussian-based Global-to-Local Mesh optimizer (GaussGLMeshizer). It leverages 3DGS as an intermediate representation to generate normal maps from both auxiliary and basic views, which serve as auxiliary priors. The global optimizer uses normal maps from auxiliary views to ensure high-fidelity detail on unknown views, while the local optimizer refines existing details using basic view normals. Moreover, a top-k mechanism is applied at different stages to mitigate prior errors, further enhancing robustness. Experiments on single view inputs demonstrate that GeometryAP outperforms state of-the-art baselines in geometry consistency, detail integrity, and structure fidelity without data training.
Huizhi Zhu, Yanping Fu, Xiangqian Shen et al.· IEEE Transactions on Visuali...· 0 citations
Multi-view 3D reconstruction undergoes several paradigm shifts over the past decades. This review categorizes the field into five representative paradigms: geometry-based Structure from Motion, learning-based Multi-view Stereo, Neural Radiance Fields, 3D Gaussian Splatting, and recent feed-forward geometric models. For each paradigm, we analyze its representation, key methods, advantages, and limitations, highlighting a clear transition from explicit geometric optimization to implicit neural representations, and further to efficient explicit modeling with pretrained feed-forward inference. Despite significant progress, challenges remain in geometric accuracy, rendering fidelity, and pose robustness. Future work is likely to focus on hybrid frameworks that combine geometric constraints, learned priors, explicit representations, and feed-forward inference to improve accuracy, efficiency, and generalization.
A semantic-guided 3D Gaussian splatting (3DGS) framework tailored to sparse-view industrial reconstruction was introduced, enabling robust reconstruction from limited viewpoints and offers a practical geometric foundation for automated inspection and remote equipment monitoring.
Boyang Li, Tianhan Gao, Zuan Gu et al.· Visual Computing for Industr...· 0 citations
Recent 3D generative models produce high-quality geometry from a single image using large-scale priors and diffusion architectures. However, they assume complete visibility and single-view inputs, limiting applicability in multi-view, occluded, or editing scenarios. Although prior works address these challenges individually, they lack a unified framework for controllable 3D completion under diverse conditioning signals. We present Axolotl3D, a multi-modal and occlusion-aware 3D generation model that jointly conditions on images, visibility masks, camera parameters, and a partial point cloud. The point cloud serves as a geometric anchor promoting faithful shape completion, while camera parameters ensure consistent multi-view alignment in a shared 3D coordinate system. A unified training strategy synthesizes diverse conditioning regimes from large-scale 3D data, enabling robust cross-modal reasoning. Experiments on Toys4K and OmniObject3D demonstrate state-of-the-art performance under both clean and occluded settings, as well as strong results in real-world reconstruction and geometry-consistent editing.
3D Gaussian Splatting (3DGS) has achieved remarkable success in novel view synthesis; however, reconstructions under sparse views often exhibit noticeable artifacts. While recent video diffusion models provide strong spatio-temporal priors for 3DGS restoration, directly fine-tuning them for restoration is suboptimal, as they lack awareness of the underlying multi-camera geometry, resulting in multi-view inconsistencies. In this work, we propose a novel 3D-aware video restoration framework designed to enhance the quality of sparse 3DGS reconstruction. Specifically, we construct a large-scale 3DGS video dataset to enable specialized fine-tuning. To bridge the gap between 2D video generation and 3D multi-view constraints, we introduce a camera-conditioned geometric prior. By using the first and last frames as boundary anchors and encoding the corresponding camera relationships, we explicitly inject spatial structure into the video generation pipeline. This boundary-anchored, camera-aware prior guides the network toward geometrically grounded restoration that remains coherent across viewpoints. Extensive experiments show that, among video-prior restoration methods, our approach attains the best pixel- and structure-level fidelity (PSNR/SSIM) and improves multi-view consistency, while remaining competitive in perceptual quality (LPIPS).
Xinhui Liu, Can Wang, Wei Jiang et al.· 0 citations
Generating photorealistic novel views from unposed images requires both 3D geometric understanding and the ability to synthesize unseen content. A natural strategy combines feed-forward 3DGS reconstruction with multi-view diffusion. Yet prior pipelines extract at most one signal from the reconstruction, either pixel rendering or learned features, while none exploits per-Gaussian visibility for occlusion-aware reference selection. This *information disconnect* leaves renderable geometry, visibility cues, and learned features unused. SplatGuide closes this disconnect by reusing a single 3DGS scene across three complementary roles. Rendered images provide pixel-aligned geometric conditioning. Per-Gaussian source-view indices are rendered into a target-view voting map for occlusion-aware reference selection. Reconstruction tokens supply feature-level guidance via cross-attention. All three signals derive from the same reconstruction forward pass. Across RealEstate10K, DL3DV, Tanks-and-Temples, and Mip-NeRF 360, SplatGuide achieves state-of-the-art pose-free novel view synthesis. On RealEstate10K, with a moderate number of input views, it surpasses the ground-truth-pose baseline.
Yejun Zhang, Zihan Wang, Xuesi Ji et al.· 0 citations