SensCluster: Sensitivity-Guided Client Clustering for Feature-Skewed Federated Learning
TL;DR
SensCluster is proposed, a novel sensitivity-aware CFL framework that constructs compact client representations by selecting parameters that are most responsive to local feature distributions, and consistently outperforms state-of-the-art CFL methods across diverse feature skew scenarios.
Abstract
Federated Learning (FL) enables collaborative model training across distributed clients without sharing raw data, but its performance degrades significantly under heterogeneous data distributions. While Clustered Federated Learning (CFL) mitigates heterogeneity by grouping similar clients, most existing methods primarily address label distribution skew and overlook feature distribution skew, where clients share labels but differ systematically in input characteristics. In this paper, we reveal a key observation: under feature distribution skew, different network layers exhibit highly unequal sensitivity, with early layers encoding strongly client-specific feature patterns that dominate aggregation behavior. Motivated by this, we propose SensCluster, a novel sensitivity-aware CFL framework that constructs compact client representations by selecting parameters that are most responsive to local feature distributions. By leveraging Fisher information, SensCluster identifies and retains only the most influential parameters, constructing sensitivity-aware representations for clustering while reducing interference from irrelevant features. Extensive experiments on both image (FEMNIST, CIFAR10-c, CINIC10-c, EuroSAT) and time-series (HAR) benchmarks demonstrate that SensCluster consistently outperforms state-of-the-art CFL methods across diverse feature skew scenarios.