Risk Equivalence between RKHS Regression and Sequence Models for Lipschitz Spectral Algorithms

Abstract

Kernel spectral algorithms are often summarized by convergence rates, which hide how their risk depends jointly on regularization, noise, the population spectrum, target coefficients, and the chosen filter. Gaussian sequence models arise as a simplified but characteristic setting for studying the interplay of these factors, where the kernel spectral algorithm corresponds to a coordinatewise shrinkage estimator. Under mild assumptions, we show that the risk of a kernel spectral estimator is asymptotically equivalent to that of the corresponding Gaussian sequence model estimator with the same filter. The explicit sequence model risk then yields a full characterization of the risk of kernel spectral algorithms in terms of the population spectrum, target coefficients, and filter. We establish this equivalence for a broad class of spectral filters, covering kernel ridge regression, generalized ridge regression, iterated kernel ridge regression, gradient flow, stable gradient descent, smoothed spectral cutoff, spectral clipping, and Pinsker shrinkage. Our risk equivalence not only holds in the classical fixed-dimensional regime but also applies to the high dimensional regime where the input dimension scales with the sample size. As applications, our risk equivalence recovers the minimax upper rates, and establishes the exact Pinsker constant in RKHS regression.