Rapid metaverse 3D assets generation via integration of neural rendering and physics based garment simulation

Abstract

Production cycles are long, and the lack of realism in virtual content is a challenge in the metaverse industry, particularly for creating immersive experiences and generating a large number of 3D assets. The current approaches rely on neural rendering, which is not physically accurate, or physics-based simulation, which needs extensive manual modeling and has unsolved trade-offs between efficiency and fidelity, and there is no clear solution to how they can be combined. To enable physically plausible 3D garment synthesis from a single-view input, this research presents a proposed method, named Neural Physics Integrated Metaverse Garment Synthesis (NP-MetaGS). Starting from a single-view image, the deformable Neural Radiance Fields first reconstruct a rough 3D structure from it with neural implicit representations. The Dynamic solvers execute dynamic simulations of virtual garments and interactions with other objects in the scene, allowing them to create highly detailed physical models including details like wrinkling fabrics and reacting to collisions. Finally, a lightweight GAN optimizes the rendering-simulation loop with cross-modal alignment loss and progressive resolution training, and Instant Neural Graphics Primitives enable neural rendering with high efficiency and depictions of high visual quality, while maintaining physical plausibility. Experimental results show higher visual metric scores, collision quality of 0.96 ± 0.02, a 93.6% reduction in penetration depth (0.012 ± 0.008 mm vs. 0.187 ± 0.092 mm for GPU-PBD) and lower error rate than conventional workflows, while also improving geometric and physical accuracy metrics.