TL;DR
This work proposes a unified transient analysis framework by embedding a sequential warm-start strategy into the radial basis function neural network (RBFNN) solver, providing a scalable pathway for uncertainty quantification and transient dynamic analysis of complex multidimensional nonlinear stochastic systems.
This work proposes Neural Kolmogorov Equations (NKEs), a deterministic, infinite-dimensional reformulation of Neural SDEs based on the Kolmogorov Forward equation, transforming the learning problem from modelling individual stochastic trajectories to modelling the evolution of probability densities.
Deterministic multiscale gas flow simulations have long suffered from the curse of dimensionality: the number of discrete velocities increases dramatically with the velocity space dimension and the Mach number, exhausting available memory and computational resources. To address this issue, this paper proposes a memory-efficient deterministic method based on an ensemble-of-subproblems strategy using stochastic discrete velocities. This strategy transforms the originally computationally expensive problem into a series of independently and efficiently solvable subproblems. To be concrete, the proposed method replaces the conventional large deterministic velocity set with multiple small random velocity sets. Each random set defines a subproblem, which is solved by a deterministic multiscale numerical scheme that computes macroscopic moments via Monte Carlo integration. The final flow field is obtained by arithmetic averaging over all sub-problems. In this work, we employ the discrete unified gas kinetic scheme (DUGKS) for spatial discretization and term the resulting method SDV-DUGKS. To validate the proposed method, several numerical test cases are conducted, including (a) the one-dimensional shock structure, (b) the two-dimensional cavity flow, and (c) supersonic flow around a square cylinder. The results of the one-dimensional shock structure confirm the feasibility of the proposed method. The two-dimensional cases demonstrate that, compared to its deterministic counterpart, the proposed method saves more than 80% of memory usage while maintaining comparable accuracy. These results indicate that the proposed method markedly reduces memory demand for multiscale flow simulations and exhibits strong potential to alleviate the curse of dimensionality that currently hinders deterministic multiscale numerical schemes from being applied to engineering problems.
Continuous-time Markov chains (CTMCs) provide the backbone for modeling discrete stochastic dynamics across applied, physical, and biological sciences. Their integration with modern gradient-based machine learning, however, is limited by the hard categorical event selection intrinsic to Gillespie-type simulation algorithms. We exploit the affine state update to obtain the exact one-step conditional-mean sensitivity by differentiating normalized reaction propensities. We pair this backward rule with exact forward trajectories to define the propensity straight-through (PST) estimator. At the trajectory level, we show that one-step sensitivities composed across events can depart from the exact multistep sensitivity. We derive the resulting per-step discrepancy in closed form and prove that it vanishes identically for affine downstream dependence. PST matches the accuracy of Gumbel-Softmax straight-through across all benchmarks: reversible dimerization (0.06% error), a genetic oscillator (1.7% error), a 50-task repressilator suite (0.17% median error), and patch-clamp ion-channel recordings ($R^2$ = 0.988). Under matched settings, PST converges 3.0-fold faster on the oscillator and 2.1-fold faster on the ion channel. At deep-learning scale, PST trains a 203,796-parameter stochastic reaction network with hard sampling, reaching 98.22% MNIST digit classification accuracy. By differentiating an exact conditional mean rather than a relaxed sample, PST offers a temperature- and Gumbel-free path to scalable gradient-based learning through exact stochastic trajectories.
We benchmark transport-based generative models as well as distillation-based few-step methods for the probabilistic forecasting of stochastic fluid flows, with a particular focus on performance under limited inference budgets. All methods are evaluated on a two-dimensional Kolmogorov flow with stochastic forcing. We measure one-step distributional accuracy against large simulated reference ensembles and assess whether the invariant measure is preserved during autoregressive rollouts via the enstrophy spectrum. On the stochastic task, flow matching achieves the most accurate one-step conditional distribution at high inference budgets, while the second-order exponential integrator DPM-2 is strongest at very low NFE. Few-step distillation methods are competitive with the multi-step methods and preserve the enstrophy spectrum particularly well. A deterministic control task, in which the forcing over the prediction interval is observed, separates aleatoric from epistemic uncertainty. Model performance does not translate between the two settings: the distilled models are competitive on the stochastic task but least accurate on the control task. While stochastic diffusion samplers such as DDPM better preserve the enstrophy spectrum during rollouts in the stochastic setting, deterministic samplers such as DDIM and DPM-2 show better spectral preservation in the deterministic setting.
Identifying stochastic dynamical systems from observational data remains a major challenge in applied mathematics and engineering, particularly when complex systems are influenced by random perturbations and incomplete empirical information. This comprehensive review aims to examine state-of-the-art data-driven methods for discovering governing equations, estimating parameters, and predicting the behavior of stochastic dynamical systems. The review systematically analyzes key methodological approaches, including Sparse Identification of Nonlinear Dynamics (SINDy), Dynamic Mode Decomposition (DMD) and its extensions, Koopman operator theory, neural ordinary differential equations, and Bayesian inference. Each approach is evaluated in terms of its theoretical foundations, computational requirements, robustness to noise, and applicability to different classes of stochastic systems. Drawing on numerical experiments and real-world case studies, the findings show that no single method consistently outperforms others across all scenarios. Instead, hybrid approaches that integrate physics-informed constraints with machine learning demonstrate the strongest potential for advancing data-driven system identification. The review concludes that future research should address real-time identification, uncertainty quantification, and the integration of multi-fidelity data sources to improve the reliability and scalability of stochastic system modeling. This work contributes a comprehensive framework for guiding researchers and practitioners in selecting and implementing appropriate identification methods for stochastic dynamical systems.
This paper investigates the distributed time-varying optimization of stochastic multi-agent systems (SMASs) using only zero-order information. Unlike existing methods that directly couple gradient estimation and optimization updates on a single time scale, this paper constructs a novel stochastic singular perturbation framework by introducing auxiliary fast systems. The proposed scheme naturally forms a slow-fast coupling structure: by introducing auxiliary variables and constructing fast subsystems to generate smooth gradient estimates, while the agent's state evolution, as the slow subsystem, performs distributed optimization and consensus. The convergence of the proposed scheme is analyzed using stochastic singular perturbation techniques and stochastic Lyapunov theory. The results show that the fast subsystem converges rapidly to the instantaneous stochastic gradient estimates, while the slow subsystem achieves practically fixed-time consensus (Pfxc) in probability and asymptotically bounded tracks the time-varying optimal trajectory. Furthermore, this paper establishes explicit bounds to characterize the effects of parameters, stochastic disturbances, and the properties of the objective function on tracking performance. Finally, the theoretical results are validated through numerical simulations.