A Cost-Efficient Pipeline for Converting 3D Gaussian Splatting Representations into Simulation-Ready Meshes for NVIDIA Omniverse
TL;DR
A practical pipeline that converts 3D Gaussian Splatting outputs into mesh-based assets and exports them into NVIDIA Omniverse and Isaac Sim workflows via OpenUSD is proposed to lower the cost of building realistic training environments while preserving visual and geometric fidelity sufficient for simulation, synthetic data generation, and robot learning.
Abstract
Neural scene reconstruction has become an important tool for building digital environments used in robotics, autonomous systems, and physical AI training. However, NeRF-based reconstruction often requires high computational cost and does not directly produce simulation-ready mesh assets. 3D Gaussian Splatting offers a faster alternative by representing a captured scene as an explicit cloud of Gaussian primitives that can be rendered in real time. This paper proposes a practical pipeline that converts 3D Gaussian Splatting outputs into mesh-based assets and exports them into NVIDIA Omniverse and Isaac Sim workflows via OpenUSD. The proposed method extracts geometry from the Gaussian primitive cloud, reconstructs a watertight or simulation-usable mesh, bakes appearance information into textures and materials, and exports the result as an OpenUSD-compatible asset with physics and semantic metadata. The goal is to lower the cost of building realistic training environments while preserving visual and geometric fidelity sufficient for simulation, synthetic data generation, and robot learning. The paper also outlines a future extension toward live conversion, where streamed captures are incrementally transformed into simulation-ready scene updates.