ViViD-GS: Voxel-Constrained Representation and Voxel-Centric Accelerator for Real-Time 3D Gaussian Splatting

Abstract

3D-Gaussian Splatting (3D-GS) enables high-quality novel view synthesis for virtual reality (VR) and augmented reality (AR) applications. Recently, Scaffold-GS is proposed to further improve rendering visual quality by generating view-conditioned Gaussians with voxel-level multi-layer perceptrons (MLPs). However, voxel-Gaussian structural mismatch and redundant MLP inference degrade rendering speed, limiting the real-time performance of Scaffold-GS on resource-constrained devices. This paper presents ViViD-GS, a voxel-centric Scaffold-GS accelerator that introduces three hardware-friendly techniques to improve rendering speed. First, we introduce voxel-constrained Gaussians and a voxel-wise rendering pipeline, which facilitates voxel-wise depth sorting and rasterization to significantly improve memory locality. Second, we propose an opacity-aware sparse MLP execution that computes parameters only for opacity-contributing Gaussians, thereby eliminating redundant inference and reducing end-to-end latency without compromising visual quality. Finally, we introduce a voxel-wise shared-scale format that shares a single exponent within each voxel to reduce parameter storage and floating-point overhead in memory-limited 3D-GS accelerators. Experimental results show that ViViD-GS improves average rendering speed by 2.28× with negligible PSNR degradation across six novel view synthesis scenes. The proposed accelerator also improves energy efficiency by up to 3.45× compared to previous 3D-GS accelerators.