NewtonGS: Physics-Structured Object-Level Neural Newtonian Dynamics for Gaussian Scene Animation

Abstract

Animating objects in a static 3D Gaussian scene requires an explicit object-level dynamic state and a controllable model of object motion. Existing dynamic Gaussian methods primarily reconstruct time-varying scenes or simulate deformation, rather than provide compact object states for direct control. To address this gap, we present NewtonGS, a physics-structured framework for object-level state rollout and Gaussian scene animation. NewtonGS represents each object with a 22-dimensional state covering pose, linear and angular velocity, anisotropic scale and its rate, mass, and contact properties. Its Gaussian Neural Newtonian Dynamics (Gaussian-NND) model combines analytic translation, quaternion kinematics, gravity, damping, and scale-restoration dynamics with learned continuous and contact residuals. A discrete event map handles floor contact. Predicted poses and scales define a shared affine transformation that updates the means and covariances of all Gaussians associated with each object. We construct two procedurally generated datasets: State-32 for state-rollout evaluation and Gaussian-32 for state-to-Gaussian transformation. On both the in-distribution and velocity-range-shift splits of State-32, NewtonGS achieves lower trajectory RMSE, final displacement error, and velocity RMSE than five analytic baselines. Experiments on Gaussian-32 further demonstrate effective conversion from predicted states to animated Gaussian objects.