Semi-analytical hierarchical Bayesian inference of nonlinear model structure in stochastic dynamics: Applied to compartmental models of infectious diseases
A Bayesian computational framework for parsimonious inference in stochastic nonlinear dynamical systems is presented, and it is shown that inducing sparsity among the model parameters eliminates redundant interactions between compartments, equivalently revealing the optimal coupling structure between differential equations.
Abstract
A Bayesian computational framework for parsimonious inference in stochastic nonlinear dynamical systems is presented. This framework enables the concurrent estimation of system states, time-varying parameters, time-invariant parameters, and the optimal sparsity structure of the model parameters. Because differential equation-based models are often simplified mechanistic or phenomenological representations, robust inference from noisy measurement data requires explicit treatment of model error and uncertainty. Model error and time-varying parameters can be represented as random processes, enabling inference while making minimal assumptions about the underlying sources of discrepancy and variability. Adopting stochastic differential equation representations affords the model significant flexibility, but can also render it susceptible to overfitting during statistical inversion, where the inferred model may track noise rather than the underlying signal. To alleviate the effects of overfitting and to enable the discovery of the optimal sparse representation of the time-invariant parameters, a Bayesian sparse learning algorithm is embedded within the framework. This sparse learning framework adopts an approximate hierarchical Bayesian setting defined by a series of semi-analytical expressions. The model structure inference framework is validated using a stochastic compartmental model for tracking and forecasting active cases of an infectious disease. Compartmental models describe population-level infectious disease dynamics through interactions among population fractions grouped by disease state. Mathematically, such models consist of a system of coupled ordinary differential equations. This example adopts an expressive compartmental model that includes multiple possible interactions between disease states, motivated by early uncertainty surrounding COVID-19 reinfection dynamics and their implications for long-term epidemic forecasting. The sparse learning exercise permits the inference of a priori unknown epidemiological dynamics from simulated public health data, discovering the nested compartmental model that optimizes the trade-off between average data-fit and model complexity. It is shown that inducing sparsity among the model parameters eliminates redundant interactions between compartments, equivalently revealing the optimal coupling structure between differential equations.
The proposed additive ODE–SDE model produces a close fit with coherent uncertainty quantification and a flexible seasonal reconstruction, while keeping the mechanistic transmission model interpretable.
Miracle Amadi, J. García-Merino, H. Haario· Bulletin of Mathematical Bio...· 0 citations
Mechanistic ordinary differential equation (ODE) models are widely used in systems biology, but uncertainty quantification (UQ) remains difficult when only a subset of state variables is experimentally observed. Existing Bayesian and likelihood-based approaches can be computationally demanding for nonlinear, weakly identifiable, or high-dimensional systems. We present a framework, and its corresponding software CUQDyn1 Plus, for UQ in partially observed ODE systems. Our method combines leave-one-out jackknife+-style empirical calibration for observed states with sensitivity-based Gaussian uncertainty propagation for hidden states. The software supports global parameter estimation, covariance propagation, bootstrap trajectory uncertainty and simulation-based calibration. It also facilitates comparison with Bayesian workflows, automated reporting, and reproducibility diagnostics. Validation on six benchmark systems shows accurate behavior in well-conditioned cases and model-dependent degradation under nonlinearity, weak identifiability, or global branch-switching non-identifiability. CUQDyn1 Plus provides a practical and computationally efficient UQ workflow for systems biology models with observed and latent states. Its diagnostic outputs help identify when local Gaussian propagation is reliable and when uncertainty bands should be interpreted cautiously, making it a useful complement to fully Bayesian workflows.
Identifying stochastic dynamical systems from observational data remains a major challenge in applied mathematics and engineering, particularly when complex systems are influenced by random perturbations and incomplete empirical information. This comprehensive review aims to examine state-of-the-art data-driven methods for discovering governing equations, estimating parameters, and predicting the behavior of stochastic dynamical systems. The review systematically analyzes key methodological approaches, including Sparse Identification of Nonlinear Dynamics (SINDy), Dynamic Mode Decomposition (DMD) and its extensions, Koopman operator theory, neural ordinary differential equations, and Bayesian inference. Each approach is evaluated in terms of its theoretical foundations, computational requirements, robustness to noise, and applicability to different classes of stochastic systems. Drawing on numerical experiments and real-world case studies, the findings show that no single method consistently outperforms others across all scenarios. Instead, hybrid approaches that integrate physics-informed constraints with machine learning demonstrate the strongest potential for advancing data-driven system identification. The review concludes that future research should address real-time identification, uncertainty quantification, and the integration of multi-fidelity data sources to improve the reliability and scalability of stochastic system modeling. This work contributes a comprehensive framework for guiding researchers and practitioners in selecting and implementing appropriate identification methods for stochastic dynamical systems.
Rishav Jha, Kameshwar Sahani, S. K. Sahani et al.· African Multidisciplinary Jo...· 0 citations
Reliable forward uncertainty quantification in engineering requires methods that account for aleatory and epistemic uncertainties. In many applications, epistemic effects arising from uncertain parameters and model form dominate prediction error and strongly influence engineering decisions. Because distinguishing and representing each source separately is often infeasible, their combined effect is typically analyzed using a unified model-error framework. Model error directly affects model credibility and predictive reliability, yet its characterization remains challenging. To address this need, we introduce a bootstrap-based stochastic subspace model for characterizing model error in the stochastic reduced-order modeling framework. Given a snapshot matrix of state vectors, the method leverages the empirical data distribution to induce a sampling distribution over principal subspaces for reduced order modeling. The resulting stochastic model enables improved characterization of model error in computational mechanics compared with existing approaches. The method offers several advantages: (1) it is assumption-free and leverages the empirical data distribution; (2) it enforces linear constraints (such as boundary conditions) by construction; (3) it requires only one hyperparameter, significantly simplifying the training process; and (4) its algorithm is straightforward to implement. We evaluate the method’s performance against existing approaches using numerical examples in computational mechanics and structural dynamics.
The Epidemic-Type Aftershock Sequence model is a well-established point process framework for characterizing earthquake occurrences. Traditionally, ETAS parameters are estimated using maximum likelihood methods, where the background rate is smoothed by a Gaussian kernel density. More recently, a Bayesian formulation has been proposed, in which the background rate is modeled as a Gaussian Process prior, enabling a flexible semi-parametric estimation. However, this approach can become computationally demanding for large datasets due to the dense covariance structure of the Gaussian Process.An alternative representation of the Gaussian Process is the Gaussian Markov Random Field, which offers a sparse precision matrix formulation that significantly reduces computational complexity. Building on this connection, we propose the Stochastic Partial Differential Equation ETAS model, where the Gaussian Process covariance matrix is replaced by a sparse precision matrix within a Log-Gaussian Cox Process framework. The precision matrix is constructed using the finite element method on a triangular mesh, and the log-prior of the Gaussian Markov Random Field is approximated via the Laplace approximation to accelerate computation.The model is first validated on synthetic datasets to assess its accuracy and computational efficiency compared to the kernel-based and Gaussian Process epidemic-Type Aftershock Sequence approaches. It is then applied to the Italian earthquake catalog (1960–2025) to estimate the stationary background seismicity with its uncertainties, relevant for seismic hazard assessment.
Sofiane Taki-Eddine Rahmani, G. Zöller, S. Hainzl et al.· Seismica· 0 citations
We introduce a new approach to Bayesian inference in potentially non-Gaussian structural vector autoregressions. It relies on the result that the elements of the impact matrix are at least set-identified with narrow bounds under standard assumptions. As a result, an efficient simulation algorithm should be capable of exploring the parameter space, even if only some (or none) of the parameters are identified. We consider very efficient Hamiltonian Monte Carlo (HMC) methods. To exploit potential deviations from Gaussianity, we recommend using a versatile error distribution, which nests a Gaussian distribution as a special case. In this manner, we can infer from the data whether the structural shocks are Gaussian and assess the strength of identification by examining the properties of the estimated shock distributions. Simulations and an empirical application to US fiscal policy demonstrate that non-identification can be easily detected from the marginal posteriors of parameters governing the shapes of the distributions of the structural shocks, even when the data are Gaussian. They also highlight the importance of efficiently accounting for non-Gaussianity.
Jetro Anttonen, Markku Lanne, Jani Luoto· Empirical Economics· 0 citations