TL;DR
This work proposes a fully Hessian-inverse-free stochastic gradient method for SMOBO which is computationally efficient due to the use of a decoupling approach, and employs a momentum-like gradient estimate to replace the stochastic gradients with their moving average, thereby mitigating the errors induced by dynamic weight updates.
A momentum-based estimator with mini-batches is employed to track the function values of each level, which are subsequently used to construct momentum gradient estimators, and a batch-free method that incorporates a first-order approximation and a clipping technique for function value estimation is developed.
We consider distributed stochastic convex optimization with a fixed budget of $N$ independent samples split among $m$ workers. Sample average approximation reduces the problem to a regularized finite-sum problem whose local Hessians are statistically similar. This allows the Hessian of the local objective at the server...
This work proposes a class of Multi-Objective Moreau Envelope based Hessian-free Algorithms (MOMEHA) to solve the multi-objective bilevel learning problems with nonconvex lower level and proposes a momentum-based variant of MOMEHA (i.e., MB-MOMEHA) method to solve the stochastic multi-objective bilevel learning problem...
This work proposes a novel single-loop algorithm based on a constrained reformulation in which lower-level stationarity is imposed as a constraint, and constructs a regularized Lagrangian by introducing a quadratic regularizer and restricting the dual variable to a bounded domain.
A line-search-free and function-value-free adaptive projected-gradient algorithm for the sample-average approximation (SAA) problem that transfers vanishing SAA residuals to Pareto stationarity for the population problem, while an additional concentration argument gives a finite-sample residual bound on compact sets.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
