We study the fundamental limits of convex-regularized estimation in high-dimensional linear regression with right-rotationally invariant design matrices. We show that the asymptotic risks of convex-penalized least squares estimators are lower bounded by the risk of an approximate message passing algorithm known as Bayes VAMP, and we further characterize when the lower bound is attainable. As a technical ingredient in the proof of our lower bound theorem, we characterize the asymptotic performance of the $\ell_2$-perturbed convex estimator for every fixed perturbation strength $\lambda>0$. This closes a gap in the literature, in which $\lambda$ was required to be sufficiently large or restrictions were imposed on the class of convex estimators. The benefit of our approach is that we can conduct a direct analysis of the spectrum's impact on the lower bound in the high-dimensional limit. In particular, we can show that the lower bound is monotone in an ordering on the limiting spectral distributions of the design matrix. These results isolate how the full singular-value distribution of the design---not merely the aspect ratio or average measurement strength---governs the limitations of convex regularization.
We consider a general model for high-dimensional empirical risk minimization whereby the data xi are d-dimensional Gaussian vectors, the model is parametrized by Θ∈Rd×k and the loss depends on the data via the projection ΘTxi. This setting covers as special cases classical statistics methods (e.g., multinomial regressi...
Kiana Asgari, Andrea Montanari, Basil Saeed· Annals of Statistics· 0 citations
We prove a quantitative central limit theorem for linear functionals of regularized empirical-risk minimizers in the proportional-dimensional regime \(p=O(n)\). The data columns are independent, not necessarily identically distributed, and satisfy a uniform columnwise Poincar\'e inequality. Under uniform curvature and...
We study the query complexity of optimization with exact scalar-value information. For globally $L$-smooth convex functions on $\mathbb R^d$ with a minimizer in a Euclidean ball of radius $R$, we prove the lower bound $\Omega(d\min\{d,\sqrt{LR^2/\varepsilon}\})$ for adaptive randomized algorithms in the stated accuracy...
Y. Dorn, D. Dvinskikh, Т. В. Логінов et al.· 0 citations
We establish complexity lower bounds for stochastic first-order algorithms in nonconvex--concave minimax optimization, allowing algorithms to use variance reduction. Our main contribution is a lower bound for a zero-respecting algorithm class that permits variance reduction, extending beyond the algorithmic restriction...
We introduce an intrinsic spectral sparsity model for nonparametric density estimation on compact connected Riemannian manifolds. Instead of penalizing coefficients in an arbitrarily chosen Laplace--Beltrami eigenbasis, we group each complete eigenspace and measure the Hilbert norm of its spectral component. The result...
This paper derives non-asymptotic quasi-optimality bounds governed by a regularized inverse Christoffel function associated with the gradient field and shows that additional smoothness of the gradient map yields sharper coherence estimates and motivates tractable importance sampling from kernel diagonal measures.
Fabio Nobile, Matteo Raviola, R. Tempone· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
