Federated learning (FL) on memory-constrained edge devices faces a dilemma: first-order (FO) optimization (i.e., backpropagation) demands substantial memory, whereas zeroth-order (ZO) optimization suffers from severe convergence slowdown. To resolve this dilemma, we introduce HO-FL, a hybrid-order FL framework that tra...
Qi-Yuan Chen, Xian Wu, Ya-Nan Ma et al.· 0 citations
Federated low-rank adaptation (LoRA) enables collaborative fine-tuning of large language models without centralizing private client data. Its factorized update, however, creates a structural mismatch in federated averaging: averaging the two LoRA factors separately does not equal averaging their products. Existing exac...
Ya-Nan Ma, Qi-Yuan Chen, Zi-Han Fang et al.· 0 citations
Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks. LLM inference mainly relies on two approaches: Autoregressive decoding (AD) generates output tokens sequentially, resulting in long latency; Speculative decoding (SD) accelerates inference...
Guan-Qiao Qu, Shuo Chen, Qian Chen et al.· 0 citations
Edge inference is a promising paradigm to provide large language model (LLM) inference services in next-generation mobile networks. LLM inference mainly relies on two approaches: Autoregressive decoding (AD) generates output tokens sequentially, resulting in long latency; Speculative decoding (SD) accelerates inference...
Fluid antenna systems (FAS) have emerged as a promising paradigm for wireless communications, enabling channel reconfigurability that offers a novel spatial degree of freedom. Nevertheless, efficiently acquiring accurate and high-resolution channel state information (CSI) in FAS remains challenging, primarily due to it...
Xue-Feng Wang, Yu-Hang Li, Yang Lu et al.· IEEE Transactions on Wireles...· 0 citations
L-shaped SFT is presented, a split fine-tuning framework that removes the need for continuous client participation and introduces one-shot SFT, in which clients upload activations once and then go offline while the server continues optimization over cached representations.
FlexP-SFT is proposed, a novel aggregation-free framework for personalized split federated fine-tuning, which fundamentally eliminates the client-side aggregation process and introduces a layer-flexible alignment strategy to balance personalization and generalization capabilities.
Jiaxiang Geng, Tianjun Yuan, Pengchao Han et al.· 3 citations
SplitLite is proposed, a communication-efficient split federated LoRA fine-tuning method that exploits the low effective rank structure of consecutive-epoch activation and gradient residuals, thereby significantly reducing both activation uplink and gradient downlink traffic.
This work establishes an efficient optimization framework for SFL under resource-constrained networks that jointly optimizes model splitting and resource allocation to minimize training cost, which is defined as the weighted sum of latency and energy costs.
Wei Wei, Xianhao Chen· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.