2025· Advances in Neural Information Processing Systems 38· 1 citation· 44 references
TL;DR
A novel FL framework is presented, FedPhoenix, that stochastically re-sets partial parameters in each round to destroy some features of the global model, guiding FL training to learn multiple generalized features for inference rather than specific overfitting features.
Abstract
Although Federated Learning (FL) is promising for privacy-preserving collaborative model training, it suffers from low inference performance due to heterogeneous client data. Due to heterogeneous data across clients, FL training easily learns client-specific overfitting features. Existing FL methods adopt coarse-grained averaging, which can easily cause the global model to get stuck in lo-cal optima, leading to poor generalization. Specifically, this paper presents a novel FL framework, FedPhoenix, to address this issue. It stochastically re-sets partial parameters in each round to destroy some features of the global model, guiding FL training to learn multiple generalized features for inference rather than specific overfitting features. Experimental results on various well-known datasets demonstrate that compared to SOTA FL methods, FedPhoenix can achieve up to 20.73% higher accuracy. The implementation is publicly available at https://github.com/UniString/FedPhoenix .
This work proposes RAVEL-FCL, a generative replay-based framework for federated continual learning that integrates an improved generative model based on Rebooting ACGAN with multi-level feature alignment to ensure consistency and employs Elastic Variational Continual Learning on the server to probabilistically regularize the global model and preserve past knowledge.
Yurui Zhou, Jia Hu, Geyong Min et al.· ACM Transactions on Autonomo...· 0 citations
Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. This dissertation presents methodologies for building efficient personalized models by identifying which strategies are effective in the global training stage and by showing how to preserve global knowledge while securing user-specific performance during local adaptation. First, we show that as data heterogeneity increases, the collapse of feature vectors is a more fundamental bottleneck than classifier weights, and propose a method that directly mitigates the discrepancy in representation magnitude between local and global models. Second, we analyze that a training approach that strengthens local alignment can induce forgetting of global knowledge (e.g., categories not observed locally), and propose a method that achieves both local alignment and global knowledge preservation by combining feature distillation based on the global model's feature vectors. Third, in federated personalized reward model learning with preference heterogeneity, we empirically verify the conventional belief that"increasing the number of global models yields better initialization,"and we show that when sufficient local fine-tuning is allowed, a single global initialization can instead provide stronger personalization performance. This study redefines the role of global initialization under data and preference heterogeneity and provides practical training strategies that simultaneously satisfy global knowledge preservation and personalization.
FedALA-WR, a novel framework that integrates adaptive attention-based aggregation with a weighted memory replay mechanism, allowing clients to reinforce valuable past knowledge during local training, consistently outperforms baseline FL and state-of-the-art pFL methods in accuracy and convergence speed.
Tinku Singh, N. Khoa, B. Kumar et al.· Cluster Computing· 0 citations
FuGuard is proposed, a dual-strategy federated unlearning framework, designed for efficient and ideal client-level data removal that combines the generative surrogate, which approximates the contribution of the target client, with optimal transport regularization that softly constrains model parameter drift during unlearning.
Pian Qi, Daniela Annunziata, Chiara Jappelli et al.· IEEE Transactions on Neural...· 0 citations
Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a federated learning framework that performs update selection in function space using an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client.
Mirko Konstantin, S. Zachow, Anirban Mukhopadhyay· 0 citations
This paper proposes Federated Learning with Consistency Optimization Algorithms (FedCO), a novel optimization framework that incorporates a label-skew-aware correction loss and neural feature distribution regularization during local training that significantly improves accuracy and convergence under diverse non-IID settings.
Ruiqi Wu, Yehong Li, Hongjie Guo et al.· Computers, Materials & C...· 0 citations