This thesis advances the training and scalability of NCDEs through three complementary contributions, building on neural rough differential equations, which reduce the time per training step for an NCDE by up to three orders of magnitude while achieving state-of-the-art performance across diverse time series benchmarks.
Abstract
Many real-world systems evolve continuously, yet most machine learning models interpret time series as discrete sequences. Continuous-time approaches instead treat time series as samples from an underlying input path, a formulation that naturally accommodates irregularly sampled or oversampled data. Among these, Neural Controlled Differential Equations (NCDEs) are a maximally expressive class of models that parametrise a vector field using a neural network and evolve their hidden state by solving a dynamical system driven by the input path. NCDEs typically use a non-linear vector field, so their expressive power and continuous-time flexibility come at the cost of a forward pass that is both computationally expensive and inherently sequential, limiting their scalability and practical applicability. This thesis advances the training and scalability of NCDEs through three complementary contributions. First, building on neural rough differential equations, Log-NCDEs apply the Log-ODE method to efficiently approximate an NCDE's solution during training, improving both computational speed and empirical performance. Second, Linear NCDEs replace the non-linear vector field with a linear one, enabling closed-form solutions and parallel-in-time computation without sacrificing theoretical expressivity. Third, Structured Linear NCDEs use structured linear vector fields to further enhance efficiency while maintaining theoretical expressiveness and empirical performance. Collectively, these methods reduce the time per training step for an NCDE by up to three orders of magnitude while achieving state-of-the-art performance across diverse time series benchmarks.
Multi-step training of sparse, interpretable models of dynamical systems directly from time-series data yields models with accurate short-term dynamics and strong agreement in long-time statistical properties, including mean, variance, and Lyapunov exponents.
One of the main objectives in control theory is to obtain a linear representation of inherently nonlinear systems in order to leverage the analytical and theoretical tools developed for linear systems. In this context, the Koopman operator has attracted increasing interest in recent years.Koopman operator theory provides a framework in which nonlinear dynamical systems are represented by a linear operator acting on an infinite-dimensional Hilbert space. Since such an infinite-dimensional representation is not numerically tractable, numerous finite-dimensional approximation methods have been proposed. These approaches typically rely on time-series data and include extended dynamic mode decomposition as well as deep learning–based variants. In this paper, we propose an original machine-learning-based approach for the synthesis of a fixed-dimensional Koopman approximant (lifting) of continuous-time nonlinear systems. A differential state-space representation of the system (as opposed to a recurrent state model) is assumed to be available through its vector field (f). The proposed encoder departs from conventional approaches in that it does not directly output the current latent state, but instead generates samples of the latent trajectory evaluated at user-defined time instants (temporal discretization). This formulation enables the integration into the learning process of Physical & Latent Continuous Losses, enforcing consistency between the physical dynamics and the Koopman dynamics, as well as Physical & Latent Boundary Losses, ensuring consistency with the prescribed initial conditions. In parallel, we introduce a structural stability constraint on the Koopman operator. The effectiveness of the proposed methodology is demonstrated through the analysis and simulation of two polynomial dynamical systems.
M. Zodros, A. Colotti, M. Yagoubi et al.· International Conference on...· 0 citations
This paper derives sufficient conditions for input-dependent contraction and formally establish an input-to-state contraction property under bounded external excitations, and develops a novel deep learning framework that seamlessly incorporates time-varying control inputs while ensuring incremental exponential convergence via input-dependent contraction regularization.
This work proposes Neural Kolmogorov Equations (NKEs), a deterministic, infinite-dimensional reformulation of Neural SDEs based on the Kolmogorov Forward equation, transforming the learning problem from modelling individual stochastic trajectories to modelling the evolution of probability densities.
Learning nonparametric systems of Ordinary Differential Equations (ODEs) from noisy data is challenging, especially when the system is input-dependent. Most current nonparametric approaches focus on autonomous systems, making them unable to capture the influence of external inputs. In this paper, we introduce a Bi-stage Gaussian Process (GP) framework for non-autonomous ODEs, capable of estimating system states and their derivatives directly from noisy measurements. The proposed method adopts a purely data-driven and nonparametric formulation, relying on Gaussian process regression and numerical integration without assuming explicit parametric system models or theoretical performance guarantees. The method is demonstrated on a scalar forced ODE with amplitudes <inline-formula> <tex-math notation="LaTeX">$A \in [{0.05, 2.5}]\pi $ </tex-math></inline-formula> and frequencies <inline-formula> <tex-math notation="LaTeX">$\omega \in [{0.1, 31.6}]$ </tex-math></inline-formula>, achieving state prediction errors below 2% for high signal-to-noise ratios (SNR = 1000) and derivative errors below 5% even for noisy measurements (SNR = 30). Furthermore, the approach is applied to a continuous stirred tank reactor (CSTR) system with inlet concentrations <inline-formula> <tex-math notation="LaTeX">$C_{A0}=1.0 2.0$ </tex-math></inline-formula> mol/m3 and flow rates <inline-formula> <tex-math notation="LaTeX">$F=0.01$ </tex-math></inline-formula> m3/s, successfully estimating reaction rates with relative errors below 4% across varying noise levels (SNR <inline-formula> <tex-math notation="LaTeX">$=100~30$ </tex-math></inline-formula>). Comparative results with non-parametric ODE (npODE), Gaussian Process ODE (GPODE) and continuous-time state-space neural network (CSNN) models demonstrate that the proposed Bi-stage GP achieves superior generalization performance under varying input conditions. The results demonstrate that the proposed method is robust, accurate, and capable of generalizing to unobserved inputs, providing a reliable alternative to classical ODE modeling in noisy and complex systems.
R. Fezai, Byanne Malluhi, N. Basha et al.· IEEE Access· 0 citations
An adaptive stability-constrained neural differential equation (AS-NDE) for systems with measured controls and unmatched, unknown perturbations and a reproducible evaluation protocol for a forced Duffing oscillator and a permanent-magnet synchronous motor model is given.
