In this paper, we investigate decentralized online convex optimization with compressed communication, where $n$ learners over a network jointly minimize a sequence of global loss functions based only on their local losses and compressed messages from their neighbors. The best known regret bounds for this problem are $O...
Si-Fan Yang, Wen-Hao Yang, Wei Jiang et al.· 0 citations
Adaptive optimization methods have received considerable attention in recent years due to their strong empirical performance. However, they generally require maintaining a preconditioning matrix, which incurs prohibitive computational overhead in large-scale applications. To address these limitations, we investigate th...
Si-Fan Yang, Yuanyu Wan, Yi-Bo Wang et al.· 0 citations
We investigate decentralized nonsmooth nonconvex stochastic optimization over a network of $n$ nodes, with the goal of finding an $(\delta,\epsilon)$-Goldstein stationary point. The best existing algorithm achieves $O(\delta^{-1}(\epsilon^{-3}+d\epsilon^{-1}))$ sample complexity and $\widetilde{O}(\gamma^{-1/2}\delta^{...
Yuanyu Wan, Lan Xue, Hao-Min Bai et al.· 0 citations
A momentum-based estimator with mini-batches is employed to track the function values of each level, which are subsequently used to construct momentum gradient estimators, and a batch-free method that incorporates a first-order approximation and a clipping technique for function value estimation is developed.
Wei Jiang, Rui Yan, Si-Fan Yang et al.· 0 citations
A framework that reduces dynamic regret minimization to switching regret minimization is presented, which can derive dynamic regret bounds by using off-the-shelf algorithms with switching regret guarantees and all the findings match the minimax optimal results for these three types of losses.
Yi-Bo Wang, Wen-Hao Yang, Si-Fan Yang et al.· 1 citation
This paper studies projection-free algorithms for stochastic constrained multi-level compositional optimization and proposes variance-reduced projection-free algorithms and establishes complexity guarantees under both the Frank-Wolfe gap and the gradient mapping criteria.
Wei Jiang, Si-Fan Yang, Wen-Hao Yang et al.· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.