On Dominant Manifolds in Reservoir Computing Networks
TL;DR
This work studies the emergence of low-dimensional dominant manifolds in the training of Reservoir Computing networks for temporal forecasting tasks and shows that the training data generate an invariant subspace of the trained reservoir, whose dimension equals the number of dominant modes.
Abstract
Understanding how training shapes the geometry of recurrent network dynamics is a central problem in time-series modeling. We study the emergence of low-dimensional dominant manifolds in the training of Reservoir Computing (RC) networks for temporal forecasting tasks. For a general linear continuous-time reservoir in the infinite-data limit, we show that the training data generate an invariant subspace of the trained reservoir, whose dimension equals the number of dominant modes. We then specialize to a simplified diagonal linear reservoir, where we link the dominant eigenvalues and eigenvectors to the spectrum of a backward Dynamic Mode Decomposition matrix, which yields a finite-dimensional approximation of the backward-time Koopman operator of the system generating the training data. We illustrate the emergence of these dominant modes during training in simulation, and discuss how the analysis may be extended to nonlinear RC via tangent dynamics and differential p-dominance.
