Preprint

Statistical inverse learning and $\ell^1$-regularization

Abhishake Rastogi T. Bubba T. Helin Luca Ratti

Jul 2026 · 0 citations · 54 references

Mathematics Computer Science

TL;DR

It is proved that membership in the approximation space $k_t$ is equivalent to polynomial decay of the best $n$-term approximation error, which is equivalent to polynomial decay of the best $n$-term approximation error.

Abstract

We study the recovery of sparse functions from finite, noisy, and indirect observations in the framework of statistical inverse learning. The unknown is modeled as an element of $\ell^1$, and observations are generated through a possibly nonlinear forward operator $A:\ell^1\to H$, where $H$ is a vector-valued reproducing kernel Hilbert space. We propose an $\ell^1$-regularized empirical risk minimizer and develop a theoretical analysis of its statistical properties. Under mild assumptions, we establish almost-sure consistency and derive non-asymptotic high-probability convergence rates in both the prediction and $\ell^1$ reconstruction norms. The rates depend on the source smoothness parameter $r$, characterized by a variational source condition, and the effective dimension exponent $b$, describing the polynomial spectral decay of the covariance operator. We further prove matching minimax lower bounds, showing that the obtained convergence rates are optimal. To relate the theory to practical sparsity models, we consider finitely smoothing operators of the form $A=G\circ S$, where $S$ is a synthesis operator, and show that approximation-space assumptions imply the required variational source conditions. In particular, we prove that membership in the approximation space $k_t$ is equivalent to polynomial decay of the best $n$-term approximation error. Finally, we verify the assumptions for two representative inverse problems: reaction coefficient identification in elliptic PDEs and sparse computed tomography. For filtered Radon transforms, we derive explicit effective-dimension asymptotics, yielding concrete convergence rates for standard image models and sparsifying systems.

arXiv Semantic Scholar

View source

Similar papers

Preprint Jul 2026

Demixing Sparse Signals from Nonlinear Observations using Generalized Non-convex Regularization

A regularization-based framework combining a Huberized data fidelity with generalized folded-concave penalties (SCAD, MCP), and a two-block proximal alternating algorithm with backtracking (NLD-PALM) whose whole iterate sequence provably converges to critical points under the Kurdyka--\L{}ojasiewicz property, with local linear rates.

Raziyeh Takbiri · 0 citations

Open access Dec 2025

Learning Lévy density via adaptive RKHS regression with bi-level optimization

Numerical experiments demonstrate that the bilevel RKHS method provides a more stable and competitive alternative to classical L-curve and generalized cross-validation strategies and that the adaptive RKHS norm is more accurate and robust than Lρ2- and ℓ2-norms for regularization.

Luxuan Yang, Fei Lu, Ting Gao et al. · Inverse Problems · 0 citations

Preprint Aug 2026

Regularization of Statistical Inverse Problems on Non-Reflexive Banach Spaces

This work investigates the stable approximation of $u^{\dagger}$ which solves the equation $Au=g$ with $A$ being a linear operator between appropriate vector spaces, with $A$ being a linear operator between appropriate vector spaces.

Darrel K. Joseph, M. Rajan · 0 citations

Preprint Aug 2026

Spectral stability of empirical metric-measure Laplacians

The variance of nonparametric estimators is typically insensitive to the regularity of the object being estimated. We establish such a property for the spectra of graph Laplacian matrices at a fixed bandwidth $h>0$. Specifically, given $n$ i.i.d. samples from a probability measure $\mu$ on a Polish metric space, we compare the eigenvalues of the empirical weighted Laplacian operator $\Delta_{\mu_n}^h$ to those of the population counterpart $\Delta_\mu^h$ under a spectral gap condition, bounding the relative error by $1/\sqrt{nv_\mu(h)}$ for eigenvalues of order smaller than $h^{-2}$, where $v_\mu(h)$ is the smallest mass of a ball of radius $h$. This bound requires very weak regularity conditions on $\mu$: it is satisfied if $\mu$ belongs to the class of coarse PI measures that we introduce. This class contains measures on metric graphs, spaces with sufficiently regular boundaries, corners, or branch points, together with discretizations or thickenings of these at scale $O(h)$. Even for measures having densities of regularity $s>2$ on manifolds (the only known case so far), our bound improves on the state-of-the-art by shaving off logarithmic factors.

Vincent Divol · 0 citations

Preprint Jul 2026

Gradient-free stochastic optimization of derivatives under strong convexity

We consider the problem of minimizing the $k$-th order partial derivative $f=\partial_j^k g$ of an unknown function $g$ along a fixed coordinate direction $j$, based on noisy queries of $g$. Assuming that $g$ has H\"older regularity ${\beta+k}$ for some $\beta\ge 2$, that $f$ is strongly convex on a compact convex set $\Theta\subset\mathbb{R}^d$ and that $g$ and $f$ satisfy mild boundedness and Lipschitz regularity conditions on $\Theta$, we propose a kernel-based estimator of $\nabla f$ and analyze the projected stochastic gradient algorithm driven by this estimator. We obtain a non-asymptotic upper bound on the optimization error of the order $d^{(2\beta+k-1)/(\beta+k)}\,N^{-(\beta-1)/(\beta+k)}$, where $N$ is the total number of queries. We also establish a minimax lower bound of the order $N^{-(\beta-1)/(\beta+k)}$ showing that this rate is optimal in $N$ over all sequential algorithms.

A. Akhavan, Sirine Louati, Alexandre B. Tsybakov · 0 citations

Preprint Aug 2026

Stochastic gradient descent with initial regularization

A variant of stochastic gradient descent with initial regularization with initial regularization is analyzed and dimension-free upper bounds on its expected excess risk for the squared loss are derived.

Nabil Kahalé · 0 citations