A novel data-driven algorithm to approximate the dominant eigenfunctions of the Koopman operator of nonlinear dynamical systems using neural networks using neural networks to fight the curse of dimensionality arising from using expressive templates for the mode approximation.
Abstract
This paper proposes a novel data-driven algorithm to approximate the dominant eigenfunctions (aka.~modes) of the Koopman operator of nonlinear dynamical systems using neural networks. The relevance of learning the dominant Koopman modes is to approximate nonlinear dynamics by linear ones in a lifted space, thereby enabling simplified control and analysis. To fight the curse of dimensionality arising from using expressive templates (here neural networks) for the mode approximation, the proposed method leverages a power-iteration scheme that directly learns the dominant Koopman modes without explicitly constructing the projection of the Koopman operator on the template of functions. Our approach connects to other approaches in the literature that avoid the curse of dimensionality by learning small dictionaries of functions, but differs from them in that we do not require ``anti-collapse mechanisms''to ensure that the learned dictionary is expressive enough to approximate the Koopman operator since our power-iteration scheme is designed to converge toward the dominant modes of the projected Koopman operator. The approach is fully data-driven, requiring only sampled state transitions. Theoretical guarantees are provided, showing convergence under increasing sample size and network width (in connection with the neural tangent kernel theorem). Numerical experiments demonstrate that the method achieves accurate and smooth approximations of dominant modes while avoiding the limitations of traditional techniques such as extended dynamic mode decomposition.
One of the main objectives in control theory is to obtain a linear representation of inherently nonlinear systems in order to leverage the analytical and theoretical tools developed for linear systems. In this context, the Koopman operator has attracted increasing interest in recent years.Koopman operator theory provides a framework in which nonlinear dynamical systems are represented by a linear operator acting on an infinite-dimensional Hilbert space. Since such an infinite-dimensional representation is not numerically tractable, numerous finite-dimensional approximation methods have been proposed. These approaches typically rely on time-series data and include extended dynamic mode decomposition as well as deep learning–based variants. In this paper, we propose an original machine-learning-based approach for the synthesis of a fixed-dimensional Koopman approximant (lifting) of continuous-time nonlinear systems. A differential state-space representation of the system (as opposed to a recurrent state model) is assumed to be available through its vector field (f). The proposed encoder departs from conventional approaches in that it does not directly output the current latent state, but instead generates samples of the latent trajectory evaluated at user-defined time instants (temporal discretization). This formulation enables the integration into the learning process of Physical & Latent Continuous Losses, enforcing consistency between the physical dynamics and the Koopman dynamics, as well as Physical & Latent Boundary Losses, ensuring consistency with the prescribed initial conditions. In parallel, we introduce a structural stability constraint on the Koopman operator. The effectiveness of the proposed methodology is demonstrated through the analysis and simulation of two polynomial dynamical systems.
M. Zodros, A. Colotti, M. Yagoubi et al.· International Conference on...· 0 citations
Fuzzy Spectral Region Decomposition (fSRD), a fully automated learning framework for estimating finite Koopman representation via multiple operators, and a data-adaptive framework for assembling locally invariant embeddings, termed Invariant Decomposition are introduced.
C. Bokor, M. Cary, Denise Morrey et al.· 0 citations
This work proposes the Fourier-enhanced alternating Levenberg--Marquardt PINN (FALM-PINN), an optimization framework that decouples representation learning from coefficient fitting within a single nonconvex optimization objective.
Yulun Wu, Matthieu Barreau, Miguel Aguiar et al.· 0 citations
Neural ordinary differential equations (ODEs) are widely used in neuroscience to model the collective activity of neurons during behavioral tasks. The high dimensionality of their parameter and activity spaces, however, often make it challenging to infer and interpret the fundamental features of their dynamics. In this study, we employ recent nonlinear dynamical system techniques to uncover the core dynamics of several Neural ODEs used in contemporary neuroscience. Specifically, using a data-driven approach, we identify Spectral Submanifolds (SSMs), i.e., low-dimensional attracting invariant manifolds tangent to the eigenspaces of fixed points. The internal dynamics of SSMs serve as nonlinear models that reduce the dimensionality of the full RNNs by orders of magnitude. Through low-dimensional, SSM-reduced models, we give mathematically precise definitions of line and ring attractors, which are intuitive concepts commonly used to explain decision-making and working memory. This unprecedented level of understanding of Neural ODEs obtained from SSM reduction enables the interpretation of mathematically well-defined and robust structures in neuronal dynamics, leading to predictions about the neural computations underlying behavior. Spectral submanifolds can be used to reduce recurrent neural networks to low-dimensional models, revealing their core dynamics. Here, authors uncover robust structures underlying decision-making and working-memory tasks, providing predictions about the underlying behavior of neural computations.
A. Marraffa, R. Krause, V. Mante et al.· Nature Communications· 0 citations
We study the population gradient flow of an infinitely wide two-layer neural network learning a misspecified single-index model in high dimension. The two layers are optimized jointly, with a perturbative parameter tuning the relative training speed between the first and second layer. This setting was considered by Berthier, Montanari and Zhou in \cite{berthier2024learning}, who conjectured a hierarchical learning scenario with explicit timescales as the second layer is trained faster than the first. In this paper, we prove that the constant and linear components of the hidden link function are indeed recovered within the predicted timescales, at sharp explicit thresholds. We then analyze the onset of learning of the quadratic component and show that the components learned at earlier stages continue to influence the dynamics in an essential way. Our proof is based on quantitative approximation results for singularly perturbed flows evolving near a manifold defined by integral constraints. At a phenomenological level, we also show that the empirical measure of the weights displays singular behaviour when reaching the quadratic component of the hidden link, with a small fraction of neurons growing significantly while the remaining ones rearrange to preserve the components already learned.
The Projected Koopman Operator Approximation framework for constructing Filtered EDMD operators is introduced, and under independent noiseless sampling and exact-rank identifiability, the resulting empirical operators converge almost surely to their population counterparts.