Reproducing Kernel (Quasi-)Banach Spaces with the $\ell^p$ Coefficient Norms

Abstract

This paper advances the theory of reproducing kernel Banach spaces (RKBSs) by investigating the general $\ell^p$ regularization framework, with a particular focus on the non-convex regime $0<p<1$. While our preceding work established the foundation for RKBSs with the $\ell^1$ norm, the pursuit of stronger sparsity---closer to the $\ell^0$ pseudo-norm---naturally motivates this exploration into $\ell^p$ quasi-norms. For $0<p<1$, we construct reproducing kernel quasi-Banach spaces (RKQBSs) with $\ell^p$ quasi-norms and establish existence and sparsity results for interpolation and regularized learning under appropriate assumptions. We characterize the linear and relaxed representer theorems in terms of a generalized Lebesgue constant and derive bounds under kernel decay, sample spacing, and stability assumptions. For $p>1$, we construct RKBSs using a fixed countable dictionary, establish minimizer existence, and identify the optimal relaxed representer factor through an off-sample coefficient operator. These results reveal fundamental differences between the regimes $0<p<1$, $p=1$, and $p>1$. Numerical experiments compare the accuracy and sparsity of solutions for these three cases.