Polyak--Ruppert averaging gives SGD and momentum methods the same asymptotic covariance. We characterize how fast finite runs reach it and how far they can be parallelized. For strongly convex quadratics with additive noise, we prove an online--offline bridge: the empirical risk minimizer on the samples of a tail avera...
Alexandra Suvroikova, D. Dvinskikh, Y. Dorn et al.· 0 citations
We study zeroth-order optimization of convex and strongly convex functions with H\"older smoothness of order $\beta>2$ from values corrupted by random noise with bias independent of the method's random directions, a Lipschitz systematic error and a bounded adversarial error. We measure the number of function evaluation...
Т. В. Логінов, D. Dvinskikh, Nazarii Tupitsa et al.· 0 citations
We study zeroth-order optimization of convex and strongly convex functions with H\"older smoothness of order $\beta>2$ from values corrupted by random noise with bias independent of the method's random directions, a Lipschitz systematic error and a bounded adversarial error. We measure the number of function evaluation...
Т. В. Логінов, D. Dvinskikh, Nazarii Tupitsa et al.· 0 citations
We study the query complexity of optimization with exact scalar-value information. For globally $L$-smooth convex functions on $\mathbb R^d$ with a minimizer in a Euclidean ball of radius $R$, we prove the lower bound $\Omega(d\min\{d,\sqrt{LR^2/\varepsilon}\})$ for adaptive randomized algorithms in the stated accuracy...
Y. Dorn, D. Dvinskikh, Т. В. Логінов et al.· 0 citations
We study entropy-regularized linear programs with sparse affine constraints and compare two exact dual representations induced by whether a redundant normalization constraint is retained or eliminated. Eliminating it yields a partially separable sum-exp dual with sparse affine factors but unbounded curvature. Our main...
I. Podlipnova, Maxim Mashtaler, A. Agafonov et al.· 0 citations
It is shown that for this mismatched problem the minimax value of the gap is of order $LR^2/N^3$ up to logarithmic factors, for deterministic and for randomized methods alike, and already in dimension proportional to the number of queries.
Т. В. Логінов, A. Gasnikov, Yuriy Dorn et al.· 0 citations
This paper proposes a novel UCB-type algorithm, M-LCB, designed to manage a pool of self-learning experts in a stochastic environment while accounting for a limited per-round learning budget and derives anytime regret bounds for M-LCB that scale with the individual regrets of the experts.
Ilgam Latypov, A. Suvorikova, Alexey Kroshnin et al.· 1 citation
This work derives a finite-sample comparison of clipped and unclipped Polyak-Ruppert averaged SGD under finite conditional $p$-th moments, and gives explicit accuracy and confidence conditions under which, for $p>2$, the Gaussian term dominates the unclipped deviation bound, so clipping need not improve its leading ord...
A. Suvorikova, E. Gladin, D. Dvinskikh et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
