A Finite Data Based Regularization in a Normed Linear Space Setting

Abstract

One of the basic problems in mathematical learning theory is to identify a function $f: \Omega\to {\mathbb R}$ with certain specific properties which fits a given training data $\{(x_i, \xi_i)\in \Omega\times {\mathbb R}: i=1, \ldots, n\}$, in the sense that, $f(x_i) = \xi_i$ for $i=1, \ldots, n$, where $\Omega$ is a compact subset of ${\mathbb R}^d$ for some $d\in {\mathbb N}$ and $f$ is required to have certain specified characteristics. We address this problem when $f$ belongs to an arbitrary normed linear space ${X}$, and the evaluation maps $f\mapsto f(x_i)$ are replaced by maps of the form $ f\mapsto \varphi_i(f)$ on $X$, where $\varphi_1, \ldots, \varphi_n$ are continuous linear functionals on ${X}$. Using an inner product structure on ${\mathbb R}^n$, we shall device a method of least-squares for obtaining an approximate solution for the above problem and also identify a subspace of ${X}$ in which the least-square solution is unique, which in turn is shown to be equivalent to solving a matrix equation. In the context when the matrix under consideration is ill-conditioned, a regularized equation is devised, and order optimal error estimates are derived for the exact targeted data $\xi=(\xi_1, \ldots, \xi_n)$ and also when it is noisy, by choosing the regularization parameter appropriately.