Aug 2026· 1 citation· ⚡ 1 influential· 20 references
Mathematics
TL;DR
It is proved that every deterministic first-order algorithm requires a first-order oracle that returns both the function value and the full subdifferential at every query point, and establishes the optimal deterministic oracle complexity.
Abstract
We study the oracle complexity of finding $\epsilon$-stationary points of $\rho$-weakly convex and $G$-Lipschitz functions, where stationarity is measured by the gradient of the Moreau envelope. We consider a first-order oracle that returns both the function value and the full subdifferential at every query point. We prove that every deterministic first-order algorithm requires $ \Omega({\rho G^2\Delta}/{\epsilon^4})$ oracle queries whenever $\Delta \leq {G^2}/{\rho}$, where $f(\bz)-\inf f \leq \Delta$. This lower bound matches the best known deterministic and stochastic first-order upper bounds, up to universal constants, and establishes the optimal deterministic oracle complexity. The result reveals a fundamental complexity separation between smooth nonconvex and nonsmooth weakly convex optimization. While smooth nonconvex minimization admits a $\Theta(\epsilon^{-2})$ oracle complexity, nonsmooth weakly convex optimization incurs an intrinsic additional $\epsilon^{-2}$ factor arising from nonsmooth geometry rather than stochasticity.
We study the deterministic first-order oracle complexity of finding stationary points of the value function in smooth nonconvex-Polyak-{\L}ojasiewicz (NC-P{\L}) minimax optimization. We assume that the objective is jointly $\ell$-smooth and satisfies the $\mu$-P{\L} condition in the dual variable, and that its value function $\Phi(x):=\max_y f(x;y)$ satisfies $\Phi(0)-\inf_x\Phi(x)\leq\Delta$. When $\kappa:=\ell/\mu\gtrsim 1$ and $0<\epsilon^2\lesssim\ell\Delta$, we prove that every deterministic first-order method requires $\Omega(\ell\Delta\kappa/\epsilon^2)$ oracle queries in the worst case to find $x$ satisfying $\|\nabla\Phi(x)\|\leq\epsilon$. This rate matches the known upper bound in its dependence on $(\ell,\Delta,\kappa,\epsilon)$ [Yang et al., 2022] and shows that the linear dependence on $\kappa$ is unavoidable for deterministic first-order methods.
A lower bound of $\Omega(\,\frac{d^2}{\log(d+1)})$ on the oracle complexity in this setting is provided, to close this gap dating back to 1996, up to polylogarithmic factors.
Whether exact scalar feedback intrinsically incurs the additional dimension $d$ paid by known zeroth-order methods remains open even for Lipschitz convex optimization. For a universal Lipschitz scale, the value only bound $O(d^2\log(d+1)\log(1/\epsilon))$ and two-point bound $O(d\epsilon^{-2})$ yield the upper bound $\widetilde O\left(d\min\{d,\epsilon^{-2}\}\right)$. By contrast, prior lower bounds for arbitrary randomized algorithms give only $\Omega(\min\{d,\epsilon^{-2}\})$, leaving a factor $d$ unexplained. We close this gap, up to logarithmic factors, for arbitrary adaptive randomized algorithms minimizing a convex objective with a universal Lipschitz scale over the $d$-dimensional Euclidean unit ball, where each query returns only the exact scalar value. Let $T_\epsilon$ denote the minimum number of queries required to return an $\epsilon$-suboptimal point with probability at least $1/2$, uniformly over the function class. We prove that \[T_\epsilon\ge c\,\frac{d\min\{d,\epsilon^{-2}\}}{\log\!\bigl(\min\{d,\epsilon^{-2}\}\bigr)},\] for $d\ge d_0$ and $0<\epsilon\le\epsilon_0$, where $c,\epsilon_0>0$ and $d_0\in\mathbb N$ are universal constants. This gives $\Omega\left(\frac{d}{\epsilon^2\log(1/\epsilon)}\right)$ in the low-accuracy regime $\epsilon\ge d^{-1/2}$ and $\Omega\left(\frac{d^2}{\log d}\right)$ in the high-accuracy regime $\epsilon\le d^{-1/2}$ with the latter independent of $\epsilon$. These bounds match the corresponding upper bound up to logarithmic factors. To our knowledge, this is the first near-optimal lower bound for arbitrary adaptive randomized algorithms throughout both accuracy regimes of exact value Lipschitz convex optimization. The proof uses a random support function hard family and develops a posterior mean energy method for adaptive exact max observations, in place of first-order zero chain constructions and noise based transcript inequalities.
Haihan Zhang, Chenheng Zhang, Zhiquan Qi et al.· 0 citations
A sharp lower bound is proved for smooth nonconvex stochastic optimization with uniformly bounded gradient noise with uniformly bounded gradient noise and resolves the question raised by whether almost-surely bounded oracle error permits a better rate than bounded variance.
We consider the problem of minimizing the $k$-th order partial derivative $f=\partial_j^k g$ of an unknown function $g$ along a fixed coordinate direction $j$, based on noisy queries of $g$. Assuming that $g$ has H\"older regularity ${\beta+k}$ for some $\beta\ge 2$, that $f$ is strongly convex on a compact convex set $\Theta\subset\mathbb{R}^d$ and that $g$ and $f$ satisfy mild boundedness and Lipschitz regularity conditions on $\Theta$, we propose a kernel-based estimator of $\nabla f$ and analyze the projected stochastic gradient algorithm driven by this estimator. We obtain a non-asymptotic upper bound on the optimization error of the order $d^{(2\beta+k-1)/(\beta+k)}\,N^{-(\beta-1)/(\beta+k)}$, where $N$ is the total number of queries. We also establish a minimax lower bound of the order $N^{-(\beta-1)/(\beta+k)}$ showing that this rate is optimal in $N$ over all sequential algorithms.
A. Akhavan, Sirine Louati, Alexandre B. Tsybakov· 0 citations
We study the first-order oracle complexity of finding a queried point with small gradient in $\ell_p$ geometry, with particular attention to the information needed to adapt the unknown smoothness and distance scales. In the strict counted local value--gradient model, no finite complexity bound can depend only on $LR/\eps$ without a nondegenerate local scale observation: a one-dimensional construction keeps $LR/\eps=4$ while defeating every prescribed finite query budget. We resolve Diakonikolas's general-$\ell_p$ parameter-free extension question for every fixed $1<p<\infty$. Under a nondegenerate secant initialization, the method knows neither the smoothness constant $L$, the initial solution distance $R$, nor $f^*$, and returns a queried point $\widehat x$ with $\|\nabla f(\widehat x)\|_q\le\eps$. For fixed finite $p>2$, we first establish the dimension-free deterministic known-parameter upper exponent $p/(p+2)$ in $K=LR/\eps$, matching the published lower polynomial exponent under its horizon and dimension qualifications. The finite local routine fits the same observable scale--radius procedure, so this exponent is preserved without knowing $L$ or $R$. Writing $\Kbar=\max\{1,LR/\eps\}$, the post-initialization pair-oracle complexity is $O_p(\Kbar^{1/2})$ for $1<p<2$, $O(\Kbar^{1/2})$ for $p=2$, and $O_p(\Kbar^{p/(p+2)})$ for $p>2$, together with the additive calibration cost $O_p(\log(e+L/M_0))$ in every regime.