We consider the problem of learning from a single finite trajectory of an ergodic stochastic dynamical system. More precisely, we study discrete-time autonomous stochastic systems defining time-homogeneous Markov processes. We first focus on estimating the optimal one-step prediction function by nonlinear least squares, and derive high-probability guarantees measured with respect to the invariant measure of the process. These results make explicit how the non-independent and non-identically distributed nature of trajectory data modifies the classical statistical learning analysis. We then extend the framework to higher-order systems and finite-state spaces. Finally, we show that the same least squares and concentration arguments naturally extend to learning Koopman operators. Our approach combines tools from statistical learning theory and quantitative ergodic theory for Markov chains. It relies, in particular, on a concentration inequality for Hilbert-space-valued additive functionals of uniformly geometrically ergodic Markov chains.
Oleksii Kachaiev, S. Villa, Lorenzo Rosasco· 0 citations
In this work, we consider the identification and control of nonlinear systems with finite action spaces. The unknown dynamics are estimated from finite samples with Koopman operator regression in a reproducing kernel Hilbert space, yielding a linear switching predictive model, the switches governed by the value of the control variable. In order to perform control in closed-loop, the learned dynamics are employed in an infinite-horizon optimal control problem with time-varying stage cost, which is solved by means of model predictive control. In a theoretical analysis, we derive learning rates for the Koopman dynamics approximation. We further quantify, under suitable assumptions, the sub-optimality of the model predictive control strategy, both in the case of exact Koopman dynamics, and in the case of learned ones. Numerical simulations on the Duffing oscillator complement our theoretical findings.
Edoardo Caldarelli, Oleksii Kachaiev, C. Molinari et al.· 0 citations
