Lightweight Cross-View Gradient Variance for Adaptive Density Control in 3D Gaussian Splatting

Abstract

3D Gaussian Splatting has become an effective representation for real-time novel view synthesis by optimizing explicit 3D Gaussian primitives from a set of posed images. A key component of this framework is adaptive density control, where Gaussians are cloned, split, or pruned during training to improve scene coverage and rendering quality. In the original formulation, density control is mainly guided by the accumulated magnitude of the view-space positional gradient. However, gradient magnitude alone does not fully describe whether different training views provide consistent geometric evidence for a Gaussian. In difficult regions, such as sparsely observed surfaces, fine structures, and ambiguous geometry, different views may push the same Gaussian in inconsistent directions, which can lead to suboptimal densification decisions. This paper presents a lightweight cross-view gradient variance strategy for adaptive density control in 3D Gaussian Splatting. Instead of relying only on gradient magnitude, the proposed method maintains an online estimate of the mean and variance of each Gaussian’s normalized gradient direction during training using a Welford-style online estimator. This variance signal is used to support more adaptive cloning, splitting, and pruning decisions. Gaussians with high gradient magnitude and low directional variance are treated as candidates for cloning, while Gaussians with high directional variance are treated as candidates for splitting. Gaussians with weak gradient response and low variance can be pruned. The proposed strategy does not require an external network, depth prior, or additional supervision, and can be integrated into the standard 3DGS optimization pipeline with minor changes. Experiments compare the proposed method against vanilla 3DGS, AbsGS, and Pixel-GS using PSNR, SSIM, and LPIPS measurements.