Skip to content
Log in
Sponsored Artificial Intelligence: A Modern Approach, Global Edition Artificial Intelligence: A Modern Approach, Global Edition
Preprint

When Is Shallow Enough? Adaptive Split Federated Learning with Client-Specific Sufficiency Estimation

Wenhao Yuan Chenchen Lin Wentao Hu Jian Chen Jinfeng Xu Shujie Li Edith C. H. Ngai
Aug 2026 · 0 citations · 59 references
Computer Science

TL;DR

A client-specific adaptation channel based on private prompt tokens, which tracks local adaptation dynamics separately from the shared backbone and provides a lightweight signal for detecting whether client adaptation remains active, and a shallow sufficiency estimator that combines cross-client semantic alignment, temporal interface stability, and prompt-state variation to estimate whether the shallowest split is already sufficient.

Abstract

\textit{Split Federated Learning} (SFL) enables distributed model training by splitting networks between the server and clients. However, under client heterogeneity, the conventional static split strategy may be suboptimal because clients can differ in data distributions, adaptation dynamics, and representation learning progress, making a single split point insufficient to accommodate client-specific training states. In this paper, we propose \textsc{FedSGA}, a \textbf{S}ufficiency-\textbf{G}uided \textbf{A}daptive split \textbf{Fed}erated learning framework that addresses this question through client-specific shallow sufficiency estimation. First, we introduce a client-specific adaptation channel based on private prompt tokens, which tracks local adaptation dynamics separately from the shared backbone and provides a lightweight signal for detecting whether client adaptation remains active. To further avoid repeated online probing over multiple candidate depths, we design a shallow sufficiency estimator that combines cross-client semantic alignment, temporal interface stability, and prompt-state variation to estimate whether the shallowest split is already sufficient. Finally, we introduce a split-compatible interface harmonization module that projects activations from different split depths into a shared semantic space, improving the comparability of heterogeneous client interfaces before server-side prediction. Extensive experiments on multiple heterogeneous benchmarks demonstrate the effectiveness of \textsc{FedSGA} in improving model performance compared with state-of-the-art methods while reducing unnecessary client-side computation.

arXiv Semantic Scholar
View source

Similar papers

Preprint Aug 2026

Capacity Confounds and Coverage Guarantees in Adaptive Sub-model Federated Learning

Sub-model federated learning lets resource-constrained clients train width-reduced versions of a global model, but existing methods allocate capacity by device resources alone. A natural next step, allocating capacity by each client's data heterogeneity as estimated from the updates the server already observes, has been repeatedly suggested. We ask whether that step is possible, using HAS-FL, an adaptive capacity-allocation framework, as a test case. Our findings are threefold. First, validated against ground-truth label-distribution divergence on reproducible partitions, update-divergence estimates of client heterogeneity are dominated by capacity rather than data: across two corrected estimators, multiple datasets, and all seeds, the estimates correlate strongly and negatively with device capacity, and no data signal remains once capacity is controlled for. This previously undocumented confound affects any method estimating client statistics from sub-model updates. Second, adaptive allocation has a hidden failure mode: when every client is capped below full width, the uncovered parameters stay at random initialization and progressively corrupt the global model. A simple coverage guarantee removes the failure and explains why uniform allocation collapses. Third, a matched-budget control settles what adaptivity contributes: random allocation to the same average budget performs no differently on both image benchmarks, and on the naturally partitioned text benchmark the adaptive policy is the weakest of the three strategies while consuming the most capacity. Sub-model training remains valuable because it admits constrained clients at quadratically reduced cost, but what protects accuracy is parameter coverage rather than allocation intelligence. Its apparent benefits come from capacity budgeting and coverage, and future designs need heterogeneity signals separable from capacity effects.

Alireza Moayedikia, A. T. Lora · 0 citations
Preprint Aug 2026

FEAST: Federated Shared-Space Training for Resource-Heterogeneous Clients

FEAST is proposed, a federated shared-space training framework that counters this imbalance by jointly training multiple subnetworks within each client's limit by introducing a one-parameter $\gamma$-allocation protocol to control this coupling.

Bostan Khan, Masoud Daneshtalab · 0 citations
Preprint Aug 2026

Coordination on a Budget: Federated Active Learning with Few Labels

A new FAL framework is proposed that utilizes federated representation learning to align client data in a shared embedding space that achieves performance that surpasses existing FAL methods even when they are given substantially larger annotation budgets, demonstrating the value of centralized coordination under privacy constraints.

Liam Mohr, D. Weinshall · 0 citations
Preprint Aug 2026

Beyond Parameter Space: NTK-Guided Personalized Aggregation for Robust Federated Learning

Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a federated learning framework that performs update selection in function space using an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client.

Mirko Konstantin, S. Zachow, Anirban Mukhopadhyay · 0 citations