VGGT-Based COLMAP-Free Initialization for Sparse-View 3D Gaussian Splatting

Abstract

Standard 3D Gaussian Splatting (3DGS) pipelines for Novel View Synthesis (NVS) are bottlenecked by Structurefrom-Motion (SfM) initialization. In casual, sparse-view scenarios, feature matching breaks down, causing the entire reconstruction process to fail. We replace this brittle dependency with a COLMAP-free, feed-forward initializer powered by a Visual Geometry Grounded Transformer (VGGT). By leveraging VGGT, our pipeline jointly estimates camera parameters and dense scene geometry across all views in a single pass. A Bridge Module then robustly normalizes the scene scale and conditions initial Gaussian opacity on geometric confidence to discourage floater artifacts during densification. Our framework reduces the initialization phase from minutes (full-scene SfM) to seconds and achieves $\mathbf{1 0 0} \boldsymbol{\%}$ initialization success from as few as three unposed images (a regime where COLMAP succeeds on only 1 of 7 Mip-NeRF 360 scenes). Project page: https://github.com/yuvanrajkrishna/VGGT-Sparse-3DGS.