This work establishes an efficient optimization framework for SFL under resource-constrained networks that jointly optimizes model splitting and resource allocation to minimize training cost, which is defined as the weighted sum of latency and energy costs.
Abstract
Split federated learning (SFL) has emerged as a powerful paradigm for model training at the edge. However, SFL inherently involves discrete decision variables for model splitting and resource allocation, resulting in a challenging mixed-integer problem. Consequently, prior optimization schemes for SFL are either \textit{heuristic} or \textit{computationally inefficient}, which cannot handle large-scale user populations. To address this limitation, this work establishes an efficient optimization framework for SFL under resource-constrained networks. Our framework jointly optimizes model splitting and resource allocation to minimize training cost, which is defined as the weighted sum of latency and energy costs. We first study the model splitting problem and develop a polynomial-time algorithm that achieves the global optimum. Then, we extend the approach to the joint model splitting and resource allocation problem. In this case, we formulate it as a two-dimensional master problem and develop an efficient approximation method with a $(1+\epsilon)$-approximation guarantee. Extensive experiments show that the proposed approach provides efficient solutions to strike the optimal energy--latency tradeoff.
Federated learning has traditionally been formulated as a single-objective optimization problem, primarily focused on maximizing model utility. In real-world applications, however, machine learning models often need to optimize multiple and potentially conflicting objectives simultaneously. This motivates federated multiobjective optimization (FMOO), which provides a natural framework for jointly handling multiple task-specific objectives in federated learning. In this paper, we propose a momentum-based variance-reduced algorithm for federated multiobjective optimization. The method incorporates a momentum-driven gradient estimator into the local updates to reduce the variance of stochastic updates, leading to an improved convergence rate. We establish theoretical guarantees showing that the expected Pareto stationarity measure of a randomly selected output iterate decays at a rate of $\mathcal{O}(T^{-2/3})$, improving upon the $\mathcal{O}(T^{-1/2})$ rates established for existing methods such as FSMGDA and FedCMOO. Numerical experiments on federated multiobjective optimization benchmarks demonstrate the effectiveness and competitive performance of the proposed algorithm.
Yong Zhao, Chunlin You, M. N. Dao et al.· 0 citations
This letter proposes Adaptive Split LoRA (AdaSplitLoRA), a framework that jointly optimizes adaptive low rank adaptor (LoRA) rank allocation and dynamic bandwidth allocation for communication-efficient split federated learning (SFL)-based large language model (LLM) fine-tuning. We formulate a joint per-round latency minimization problem over server-side LoRA ranks and uplink bandwidth, and decompose it into two independent subproblems. For server-side rank adaptation, we employ a gradient-based importance heuristic to address the inherent intractability of the discrete rank optimization. For uplink bandwidth allocation, we design a min–max straggler latency problem, prove its convexity, and obtain the global optimum efficiently using interior-point methods. Experimental results show that AdaSplitLoRA outperforms or remains competitive with fixed/adaptive-rank baselines while using fewer server-side adapter parameters and achieving favorable accuracy–latency tradeoffs in heterogeneous wireless SFL settings.
HyunJun Choi, Joohyung Lee, Ronald Y. Chang· IEEE Wireless Communications...· 0 citations
Federated learning (FL) enables privacy-preserving distributed model training but faces challenges from heterogeneous model architectures and limited communication resources at the network edge. Federated knowledge distillation (FedKD) alleviates model heterogeneity by combining prototype-wise parameter aggregation and knowledge transfer across heterogeneous models. However, transmitting gradients still introduces considerable communication overhead, while existing compression approaches typically apply a uniform strategy across clients and ignore their diverse model characteristics and resource capacities. To address this issue, we propose a heterogeneous compression framework for FedKD that enables each client to select a compression strategy from a candidate strategy set. We formulate the compression strategy selection problem as a non-stationary stochastic multi-armed bandit (MAB), where each arm corresponds to a compression strategy. An efficiency-aware reward is designed by jointly considering local optimization improvement, global knowledge alignment, and execution time. Based on this formulation, we develop an Adaptive heterogeneouS Compression algorithm for fEderated kNowledge Distillation (ASCEND), which employs an exponential moving average (EMA)-enhanced $\epsilon$-greedy policy to balance exploration and exploitation. Experimental results on multiple datasets demonstrate that ASCEND effectively adapts to heterogeneous model and resource settings, reducing communication overhead and training time while maintaining competitive model accuracy.
Chen-Wang Liu, Yijun Liu, Chang Liu et al.· 0 citations
FEAST is proposed, a federated shared-space training framework that counters this imbalance by jointly training multiple subnetworks within each client's limit by introducing a one-parameter $\gamma$-allocation protocol to control this coupling.
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
MIT News · Artificial Intelligence· news.mit.eduAug 24, 2026
