This paper considers the design of optimal fixed-step first-order methods for high-dimensional minimization of $L$-smooth convex functions. For optimizing worst-case performance measured via suboptimality of the final function value (relative to the initial squared distance to a minimizer), we provide an algebraic proof of the optimality of the optimized gradient method (OGM) and establish its uniqueness among all fixed-step first-order methods. For the alternative measure of final squared gradient norm (relative to initial suboptimality), we prove the OGM-G method is optimal and uniquely so among fixed-step first-order methods. Finally, for the setting measuring the final squared gradient norm (relative to the initial squared distance to a minimizer), we show the recently proposed Lemniscate method is optimal and uniquely so. Our proofs rely on algebraic reductions for lower bound arguments rather than traditional information-theoretic bounds, which were previously only able to establish OGM's optimality but not uniqueness.
We determine the exact worst-case function-value suboptimality after $N$ first-order oracle calls on $L$-smooth, $\mu$-strongly convex functions under a nonnegative weighted combination of the initial squared distance, function-value suboptimality, and squared gradient norm. Excluding the case where no strict improveme...
It is proved that if $\mathcal G_N(\mathcal A)$ denotes the bound on the squared gradient norm after $N$ iterations for a method $\mathcal A, it is proved that $\limsup_{N\to\infty}N \mathcal G_N(\mathcal A) \ge 1/2$ for any method $\mathcal A$.
We provide two complementary explanations of H-duality in smooth strongly convex optimization and contractive fixed-point problems. H-duality refers to the phenomenon where the worst-case performance of many fixed-step first-order methods (FSFOMs) for a given performance setup are exactly equal to the worst-case perfor...
Minimizing gradients of a convex function is an important problem across optimization and learning tasks. The gradient provides a directly computable certificate of approximate stationarity, and its minimization usually implies stronger results than those for minimization of function values. In this work, we study grad...
Nico Pelleriti, Maryam Shiran, David Martínez-Rubio et al.· 0 citations
In this paper we propose first-order methods for a class of nonsmooth composite strongly convex--strongly concave and nonconvex--concave minimax optimization. We first develop an inexact proximal method and an accumulative regularizated method for strongly convex--strongly concave problems. The latter achieves the opti...
The same accuracy exponent holds beyond tensor update rules, even for randomized queries and arbitrary feasible outputs, for smooth convex--concave minimax optimization.
Yan-Yi Li, Hai-Han Zhang, Chen-Heng Zhang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
