We prove a quantitative central limit theorem for linear functionals of regularized empirical-risk minimizers in the proportional-dimensional regime \(p=O(n)\). The data columns are independent, not necessarily identically distributed, and satisfy a uniform columnwise Poincar\'e inequality. Under uniform curvature and smoothness assumptions, and for a quadratic regularizer, we show that every nondegenerate statistic \(\sqrt n\,u^\top\hat\theta\), centered by its expectation and normalized by its standard deviation, converges to a standard normal random variable in Wasserstein distance, with rate \(O((\log n)^7n^{-1/4})\). The proof is based on moment and stability bounds for the minimizer, a second-order leave-one-out expansion, and a perturbative normal-approximation argument for functions of independent variables. We also prove the variance upper bound \(\Var(u^\top\hat\theta)\le C\norm{u}_2^2/n\), identifying the \(\sqrt n\) fluctuation scale.
Rates of convergence in normal approximation are fundamental to probability and statistics. The theory has evolved from normalized sums to Studentized statistics, smooth functions of sample means, and $U$-statistics, and more generally to symmetric statistics. A central question throughout this development has been to...
Bing-Yi Jing, Yi-Ming Liu, Shao-Chen Wang et al.· 0 citations
Let $Y_1,\ldots,Y_n$ be independent symmetric random variables with log-concave tails. We give a dimension-free characterization of the expected supremum of the canonical process $X_x=\sum_{i=1}^n x_iY_i$ without any $\Delta_2$ or regular-growth assumption on the coordinate tails. The characterization is governed by sc...
Uniform stability is a classical tool for controlling the generalization error of a learning algorithm. Bousquet, Klochkov, and Zhivotovskiy (2020) showed that the problem can be reduced to a moment inequality for a sum of weakly interacting functions of independent random variables. Their bound contains an additional...
We prove Breiman's conjecture under the first-moment assumption. Let $Y_1,Y_2,\ldots$ be iid nonnegative random variables with $\mathbb P\{Y_1>0\}>0$, normalized by their sum. If the resulting randomly weighted sum converges to a nondegenerate law for one fixed integrable, nonconstant mark distribution, then the tail o...
We consider a general model for high-dimensional empirical risk minimization whereby the data xi are d-dimensional Gaussian vectors, the model is parametrized by Θ∈Rd×k and the loss depends on the data via the projection ΘTxi. This setting covers as special cases classical statistics methods (e.g., multinomial regressi...
Kiana Asgari, Andrea Montanari, Basil Saeed· Annals of Statistics· 0 citations
We consider the problem of noisy gradient-free minimization of the k-th order partial derivative of a $\beta$-H{\"o}lder function supported on a d-dimensional cube. We show that T ^{($\beta$+d+k)/(2$\beta$+d)} log(T )^{(\beta-k)/(2\beta+d)} is a non-asymptotic minimax rate of the T step cumulative regret for all $\beta...
Théo Paquier, A. Tsybakov, F. Portier et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
