In this paper, we investigate decentralized online convex optimization with compressed communication, where $n$ learners over a network jointly minimize a sequence of global loss functions based only on their local losses and compressed messages from their neighbors. The best known regret bounds for this problem are $O...
Si-Fan Yang, Wen-Hao Yang, Wei Jiang et al.· 0 citations
Adaptive optimization methods have received considerable attention in recent years due to their strong empirical performance. However, they generally require maintaining a preconditioning matrix, which incurs prohibitive computational overhead in large-scale applications. To address these limitations, we investigate th...
Si-Fan Yang, Yuanyu Wan, Yi-Bo Wang et al.· 0 citations
A momentum-based estimator with mini-batches is employed to track the function values of each level, which are subsequently used to construct momentum gradient estimators, and a batch-free method that incorporates a first-order approximation and a clipping technique for function value estimation is developed.
Wei Jiang, Rui Yan, Si-Fan Yang et al.· 0 citations
Agnostic federated learning (AFL) seeks a model that performs reliably across $m$ heterogeneous workers, but communication remains a bottleneck. We improve communication efficiency by reducing the number of synchronization rounds via faster convergence and the communication cost per round via compression. We first prop...
Hao-Min Bai, Jun-Yan Sun, Si-Fan Yang et al.· 0 citations
A framework that reduces dynamic regret minimization to switching regret minimization is presented, which can derive dynamic regret bounds by using off-the-shelf algorithms with switching regret guarantees and all the findings match the minimax optimal results for these three types of losses.
Yi-Bo Wang, Wen-Hao Yang, Si-Fan Yang et al.· 1 citation
This paper studies projection-free algorithms for stochastic constrained multi-level compositional optimization and proposes variance-reduced projection-free algorithms and establishes complexity guarantees under both the Frank-Wolfe gap and the gradient mapping criteria.
Wei Jiang, Si-Fan Yang, Wen-Hao Yang et al.· 3 citations
Traditional variance reduction methods (e.g., SPIDER, SARAH, STORM) have been extensively investigated for improving the convergence rates of stochastic optimization. These techniques typically maintain a sequence of estimators for a single function (or gradient) across iterations. However, what if we need to track mul...
Wei Jiang, Si-Fan Yang, Yi-Bo Wang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.