We study a deterministic family of sharpness-aware minimization methods for smooth nonconvex functions. The perturbation is $$ y_k=x_k+\rho\, \frac{\nabla f(x_k)}{\norm{\nabla f(x_k)}^\alpha}, \qquad 0\leq\alpha\leq 1, $$ so that its effective radius is $\rho\norm{\nabla f(x_k)}^{1-\alpha}$. For $0<\alpha\leq1$, we give an explicit complexity bound above the stationarity level $(L\rho)^{1/\alpha}$. A one-dimensional quadratic example reaches this level exactly, showing that the bound describes a real limitation of the constant-parameter rule. The unnormalized case $\alpha=0$ is treated separately and requires $L\rho<1$. We then introduce a clipped rule which agrees with the constant-$\rho$ rule away from stationary points and becomes proportional to the gradient near them. The clipped method has $\norm{\nabla f(x_k)}\to0$ and the usual $O(T^{-1/2})$ stationarity bound. Numerical tests on a quadratic function, the Rosenbrock function, and a five-dimensional nonconvex function illustrate the stationarity floor of the unclipped rule and the effect of clipping.
It is proved that every deterministic first-order method requires $\Omega(\ell\Delta\kappa/\epsilon^2)$ oracle queries in the worst case to find $x$ satisfying $\Phi(0)-\inf_x\Phi(x)$ and that the linear dependence on $\kappa$ is unavoidable for deterministic first-order methods.
Let $\alpha$ be irrational and let $q_j$ be the denominators of its continued-fraction convergents. We study the function \[ h(x)=\sum_{j\geq1}\frac{\cos(2\pi q_jx)}{q_j} \] and the skew product \[f(x,y)=(x+\alpha,y+h(x))\quad\mathrm{mod}\quad\mathbb{Z}^2.\] The function $h$ is H\"older continuous of every exponent bel...
We study lower bounds on the first-order oracle complexity of smooth nonconvex-concave minimax optimization. We consider objectives $f$ that are jointly $L$-smooth in the primal and dual variables $(x,y)$, concave in $y$, and whose primal value function $\Phi(x) := \max_{y\in\mathcal Y} f(x,y)$ satisfies the initial-ga...
A tight lower bound is proved of $\Omega(\lambda^{1/2}\epsilon^{-7/2})$ for finding points satisfying the recently proposed relaxed notion of $(\lambda,\epsilon)$-stationarity, which allows combining further-away gradients.
We prove a joint stochastic zeroth-order lower bound for Goldstein stationarity on a Euclidean query ball, even when the ball is guaranteed to contain a stationary point. In dimension $d$, let $f=\mathbb{E}[F(\cdot;\xi)]$, assume $\mathbb{E}[\operatorname{Lip}(F(\cdot;\xi))^2]\le L_0^2$, and bound the initial objective...
For $N\geq 1$, $p\geq 1$, and $\delta>0$, consider the nonlocal threshold functional \[ I_{\delta,p}(u)=\iint_{\{|u(x)-u(y)|>\delta\}} \frac{\delta^p}{|x-y|^{N+p}}\,\dd x\,\dd y. \] Nguyen and Squassina \cite{NguyenSquassina} asked whether $I_{\delta,p}$ decreases under Schwarz rearrangement. We give a negative answer...
Qinyang Li· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
