This work introduces a probabilistic, non-intrusive reduced-order model (ROM) for chaotic dynamical systems, arguing that projecting high-dimensional nonlinear dynamics onto a low-dimensional manifold introduces irreducible uncertainty, compounded by the chaotic attractors and multi-admissible futures inherent to turbulent flows.
Abstract
Stochastic Differential Equations (SDEs) have become a cornerstone of scientific machine learning, though they are predominantly utilized as algorithmic tools for uncertainty quantification or distribution matching. In contrast, leveraging SDEs fundamentally to model macroscopic, nonlinear physics as stochastic processes remains largely unexplored. This work introduces a probabilistic, non-intrusive reduced-order model (ROM) for chaotic dynamical systems. We argue that projecting high-dimensional nonlinear dynamics onto a low-dimensional manifold introduces irreducible uncertainty, compounded by the chaotic attractors and multi-admissible futures inherent to turbulent flows. Consequently, a chaotic system governed by a partial differential equation can be effectively modeled by an SDE in a suitable latent space. To this end, a nonlinear autoencoder is employed to map the flow field into a low-dimensional representation, within which the temporal evolution is explicitly governed by an SDE. The predictable component of the dynamics is captured by a learned drift term, while state-dependent stochasticity is absorbed by a diffusion term. We demonstrate that this probabilistic framework successfully propagates highly nonlinear states, offering a robust alternative to traditional deterministic methodologies for chaotic regimes. Ultimately, our model generates new chaotic flow trajectories that remain locally and globally consistent with the true transition kernel learned from Direct Numerical Simulation (DNS) data. Even though these generated trajectories are unique and distinct from the training set, they preserve the underlying statistics and manifolds, validating the strong generative performance and robustness of our methodology.
This work proposes Neural Kolmogorov Equations (NKEs), a deterministic, infinite-dimensional reformulation of Neural SDEs based on the Kolmogorov Forward equation, transforming the learning problem from modelling individual stochastic trajectories to modelling the evolution of probability densities.
Deep learning surrogates for forecasting chaotic dynamical systems suffer from catastrophic error accumulation over long-term autoregressive rollouts. This behavior is partly tied to the underlying systems: chaotic spatiotemporal systems, such as the Kuramoto-Sivashinsky (KS) equation, visit phase space unevenly - dominated by recurrent, low-dimensional quiescent states (e.g., near-laminar flows) and punctuated by rare, dynamically complex topological transitions (e.g., wave-merging events). Under a sample-wise uniform objective, standard neural surrogates allocate their finite capacity to the statistically numerous quiescent states, under-representing the transient regimes that trigger disproportionate, localized errors. Existing imbalanced-regression methods reweight samples by target-space density. However, statistical target-space rarity need not coincide with the intrinsic dynamical rarity - the recurrence geometry of the attractor that is the source of the imbalance. To address this, we introduce Dynamics-Aware Weighting (DAW), a data-centric objective reweighting framework. Using the local dimension $d$ from dynamical systems theory as an a priori measure of a state's active degrees of freedom, DAW reshapes the loss landscape to allocate representational capacity toward the sparse, high-$d$ regimes where forecast errors are systematically large. On the chaotic KS equation, DAW consistently outperforms uniform training, purely statistical density weighting, and its randomly permuted ablation, reducing long-term autoregressive error relative to all baselines. Event-level analysis shows that DAW achieves this by suppressing the localized error amplifications incurred during sharp jumps in $d$, which accompany complex physical processes such as wave-merging in the KS system.
Anomalous diffusion in complex systems, such as intracellular transport and soft glassy materials, often emerges from complex dynamics between a tracer particle and multiple internal states of its heterogeneous environment. We develop a deep learning framework based on backward stochastic differential equations to solve high-dimensional anomalous Fokker-Planck equations with numerous internal states in an unbounded domain. The transition matrices of internal states can be singular or rectangular. Tailored algorithmic strategies are proposed for each case. The resulting scalable algorithms provide a way to probe the microscopic origins of anomalous diffusion in high-dimensional heterogeneous systems. Their effectiveness and accuracy are validated in several ways, depending on the number and dimension of the equations to be solved.
From a modeling perspective, such systems are naturally described by high-dimensional coupled Langevin equations or their associated Fokker-Planck (FP) equations [12,13]. The internal states, which represent distinct dynamical modes or local environments, are governed by a transition matrix. A significant mathematical and computational challenge arises since this matrix can be singular or non-singular, reflecting different physical scenarios such as absorbing states or transitions between different environments. The high-dimensionality of the state space and the potentially singular transition matrix make the problem intractable for traditional numerical methods. To address these challenges, we propose a novel deep learning framework based on backward stochastic differential equations (BSDEs) to solve high-dimensional anomalous Fokker-Planck equations with numerous internal states in an unbounded domain. The framework is designed to handle both singular and non-singular transition matrices, providing a unified approach to a broad class of anomalous diffusion problems. The proposed algorithms are scalable and can effectively probe the microscopic origins of anomalous diffusion in high-dimensional heterogeneous systems. We validate the effectiveness and accuracy of our algorithms through extensive numerical experiments, demonstrating their capability to recover known solutions for intractable lower-dimensional systems and by exploring previously inaccessible regimes and comparing with the solutions obtained through other deep learning approaches in high dimensions. Ultimately, this scalable methodology not only provides a powerful computational tool but also serves to probe the microscopic origins of anomalous diffusion by enabling the direct interrogation of complex, high-dimensional coupled dynamics.
Weihua Deng, Sidra Abid Kayani, Yongtao Shi et al.· Journal of Computational Mat...· 0 citations
A framework is developed for the inference of dynamics described by a generalized system of ordinary differential equations. A stochastic gradient method is coined that infers dynamics from observed marginal probability density functions using the joint probability density function of the observable and latent variables. Diffusion and other irreversible processes observed in a low-dimensional state can be recast as deterministic, reversible flows in a sufficiently augmented state space, where the joint density satisfies the hyperbolic Liouville equation. The marginal distribution observed is the projection of these hyperbolic dynamics onto the observed coordinates, with the latent components carrying the randomness and memory. This reframing allows inference for irreversible or stochastic dynamics into the recovery of a deterministic Ordinary Differential Equation (ODE) from marginal observations. Instead of solving the high-dimensional Liouville equation for the joint density, the algorithm exploits its characteristic representation. Particles sampled from the initial distribution are transported along characteristic lines. The Eulerian sensitivity with respect to parameters is obtained by sensitivity propagation along the characteristic lines, with a crossed U-statistic producing an unbiased gradient estimator, which enables stochastic gradient descent. Four experiments validate the method: recovery of a three-mode linear system observed through the marginal of a single mode; a nonlinear Gompertz growth model with a hidden mode; a bistable system whose hidden mode turns a unimodal marginal bimodal; and Stokes--Oseen drag law recovery for particles in a cellular flow. Convergence behavior is analyzed across these settings.
A multi-regime RC framework in which multiple readouts are trained under different dynamical conditions and combined through a short observation window to form a trajectory-dependent linear readout enables both regime identification and adaptation to unseen or intermediate dynamics.
S. Hadipour Lakmesari, H. Kantz, Francesco Sorrentino· Chaos· 0 citations
This work proposes a unified transient analysis framework by embedding a sequential warm-start strategy into the radial basis function neural network (RBFNN) solver, providing a scalable pathway for uncertainty quantification and transient dynamic analysis of complex multidimensional nonlinear stochastic systems.
Zi Yuan, Lincong Chen· International Journal of Dyn...· 0 citations