We characterize the oracle complexity of finding first-order stationary points in smooth nonconvex--strongly-concave (NC--SC) minimax optimization when each query provides the full Hessian in addition to the function value and gradient. For $\Phi(x)=\max_y f(x,y)$, assume that $f$ has an $L$-Lipschitz gradient, a $\rho...
Fang-Qing Zhang, Hai-Han Zhang, Wen-Dao Wu et al.· 0 citations
Does randomization improve the worst-case first-order complexity of nonconvex optimization when the objective is very smooth? We prove that, for every fixed finite integer $p\ge3$, finding an $\eps$-stationary point requires $\Omega_p(\eps^{-5/3})$ exact value-and-gradient queries, even for $C^\infty$ objectives with i...
Hai-Han Zhang, Wen-Dao Wu, Fang-Qing Zhang et al.· 0 citations
We study the sample complexity of finding a point with expected hypergradient norm at most $\epsilon>0$ in nonconvex--strongly-convex stochastic bilevel optimization using fresh, globally unbiased first-order samples with bounded variance. For lower-variable smoothness order $p\ge1$, F$^2$SA-$p$ achieves $\widetilde O(...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 0 citations
We study deterministic adaptive optimization of globally $\beta$-smooth, $\mu$-strongly convex functions using exact scalar function values. Queries and outputs lie in $B_2^d(R)$, and the minimizer lies in $B_2^d(R/2)$. Set $\kappa=\beta/\mu$, $Q=\beta R^2/\epsilon$, and $D_d=(d/\log(ed))^{1/3}$. For sufficiently large...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 0 citations
We establish a randomized first-order lower bound for finding an $\epsilon$-stationary point, $\|\nabla f(x)\|\le \epsilon$, of a nonconvex function with initial gap at most $\Delta$, $L_1$-Lipschitz gradient, and $L_2$-Lipschitz Hessian. Each oracle call returns the exact function value and gradient. Let $Q_{\mathrm{r...
Hai-Han Zhang, Wen-Dao Wu, Chen-Heng Zhang et al.· 0 citations
We prove a joint stochastic zeroth-order lower bound for Goldstein stationarity on a Euclidean query ball, even when the ball is guaranteed to contain a stationary point. In dimension $d$, let $f=\mathbb{E}[F(\cdot;\xi)]$, assume $\mathbb{E}[\operatorname{Lip}(F(\cdot;\xi))^2]\le L_0^2$, and bound the initial objective...
We establish tight randomized higher-order oracle complexity for finding first-order stationary points of nonconvex finite sums. Let $n$ be the number of components, $\Delta>0$ the initial objective-gap bound, $L_p>0$ an individual $p$-th derivative Lipschitz bound, and $\epsilon>0$ the target gradient norm. For every...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 1 citation
We characterize the fresh-gradient oracle complexity of smooth nonconvex-strongly-concave minimax optimization, with matching upper and lower bounds up to logarithmic factors. Let $\Phi(x)=\max_y f(x,y)$, where $f$ is jointly $L$-smooth and $\mu$-strongly concave in $y$ on unconstrained Euclidean domains, and set $\kap...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 0 citations
We establish matching polynomial query bounds for low-rank approximation from exact matrix--vector products. Given an unknown matrix $A\in\mathbb{R}^{m\times n}$, at each step a randomized algorithm chooses either $v\in\mathbb{R}^n$ and receives $Av$, or $u\in\mathbb{R}^m$ and receives $A^\top u$. The choice may depend...
Hai-Han Zhang, Wen-Dao Wu, Chen-Heng Zhang et al.· 1 citation
The same accuracy exponent holds beyond tensor update rules, even for randomized queries and arbitrary feasible outputs, for smooth convex--concave minimax optimization.
Yan-Yi Li, Hai-Han Zhang, Chen-Heng Zhang et al.· 0 citations
A matching $\Omega_p(Q^{2/(3p-1)})$ lower bound is proved for arbitrary adaptive deterministic algorithms and randomized algorithms with per-instance success probability at least $2/3$, without span or tensor-update restrictions.
A single fixed smooth convex hard instance is constructed using a Moreau-smoothed biased max chain, an exact prefix-shielding mechanism, and batched delayed rotations to preserve consistency with the full adaptive transcript and establish the optimality of the square-root complexity branch for deterministic bounded-que...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
