Aug 2026· Journal of Scientific Computing· Vol 109· 0 citations· 33 references
Computer Science
TL;DR
A zeroth-order stochastic proximal Newton-type algorithm with a new norm test for adaptive sample size selection is proposed and both iteration complexity and oracle complexity of the algorithm are established for nonconvex and strongly convex objectives, respectively.
It is proved under reasonable assumptions that the stochastic proximal-gradient method reaches an $\epsilon$-KKT point within $O(\epsilon^{-2})$ iterations with high probability and implies almost-sure termination in finitely many iterations and an $O(\epsilon^{-2})$ bound on the expected iteration complexity.
F. Curtis, Xiaoyi Qu, Daniel P. Robinson· 0 citations
For the method of successive approximations (MSA), a fundamental challenge is that the stochastic maximum principle (SMP) provides an optimality condition rather than a descent rule. To overcome this difficulty for nonconvex stochastic control problems, we develop a soft-update MSA (SU-MSA) with a second-order-adjoint...
In this paper, we study a penalty method for simple bilevel optimization in which the lower-level objective has a convex composite structure. Our approach is based on a vanishing-penalty reformulation, which gives rise to a sequence of penalized problems as the penalty parameter decreases. A key difficulty is that metr...
Xiao-Ning Bai, Boris S. Mordukhovich, Shang-Zhi Zeng et al.· 0 citations
This paper proposes a new method, which combines the gradient method using the approximately optimal stepsizes with the Barzilai and Borwein scaling conjugate gradient method, and propose two scaling conjugate gradient methods for nonlinear unconstrained optimization problems. And then, the sufficient descent property...
Yongjin Kim, Yunchol Jong, Pyongil Choe· Journal of Decision Science...· 0 citations
In this work, we develop a zeroth-order (ZO) method that estimates the gradient with minimal bias and variance. Instead of the random directions often used in the ZO literature, we consider central differences with directions given by a fixed spherical $t$-design, a finite point set that integrates every polynomial of...
Amir Ali Farzin, Yuen-Man Pun, Philipp Braun et al.· 0 citations