2026· Computers, Materials & Continua· 0 citations· 31 references
TL;DR
This paper proposes Federated Learning with Consistency Optimization Algorithms (FedCO), a novel optimization framework that incorporates a label-skew-aware correction loss and neural feature distribution regularization during local training that significantly improves accuracy and convergence under diverse non-IID settings.
Abstract
: Federated learning (FL) enables collaborative training of deep neural architectures while preserving data privacy, yet its performance often deteriorates in non-IID scenarios, which stems from client-side distribution drift and divergent local updates induced by pervasive data heterogeneity. This challenge is particularly critical for maintaining the structural consistency and generalization of neural models across diverse, distributed sources with significant distribution shifts. In this paper, we investigate how to effectively mitigate label distribution shift and feature distribution skew to enhance the global representation stability of neural architectures. We propose Federated Learning with Consistency Optimization Algorithms (FedCO), a novel optimization framework that incorporates a label-skew-aware correction loss and neural feature distribution regularization during local training. Specifically, our method aligns the internal statistics of architectural components between local and global models to suppress feature-space drift. Combined with an adaptive global aggregation mechanism guided by label entropy, this approach ensures that model updates from heterogeneous clients are consistently integrated into the global architectural parameters. Experimental results on multiple benchmarks demonstrate that FedCO significantly improves accuracy and convergence under diverse non-IID settings. For instance, on CIFAR-10 with extreme heterogeneity ( α = 0.05), FedCO achieves 73.64% accuracy, outperforming FedAvg by 7.32%; it also attains the highest accuracies on CIFAR-100 (62.63%) and TinyIma-geNet (37.82%) under the same setting. Our findings provide a robust strategy for integrating distribution-aware local training with adaptive structural aggregation, offering new insights into enhancing the reliability of distributed neural systems in real-world deployments. The code related to this algorithm is available at https://github.com/Donglin0730/ FedCO.
This work proposes RAVEL-FCL, a generative replay-based framework for federated continual learning that integrates an improved generative model based on Rebooting ACGAN with multi-level feature alignment to ensure consistency and employs Elastic Variational Continual Learning on the server to probabilistically regularize the global model and preserve past knowledge.
Yurui Zhou, Jia Hu, Geyong Min et al.· ACM Transactions on Autonomo...· 0 citations
Federated learning is a novel distributed machine learning framework with privacy-protection, yet it is vulnerable to the effects of heterogeneous data. Heterogeneous data drive client models that overfit local datasets and depart from the global optimum during local training, which is termed client drift. To address the impact of client drift, we approach this issue from the perspectives of optimization and generalization. We comprehensively considering the effects of client drift during the training process, and quantifying it as the aggregation error. We first propose adaptive gradient regularization, which is based on gradient regularization and further and applies different regularization strengths to each parameter based on the magnitude of the parameter variance between the local model and the global model, thereby mitigating the performance degradation caused by aggregation error and helping model converge to a flatter minimum. In order to obtain the variance between local and global models to compute our adaptive gradient regularization term, we introduce global momentum from the server side as the approximation of global gradient and further utilize it as a gradient correction term. Next, we propose MoFedAGR, which combines gradient correction term and adaptive gradient regularization term, helping client models converge to a consistent flat minimum. We have provided the theoretical convergence bounds of the algorithm we proposed. Furthermore, experiments on several image classification datasets demonstrate that our algorithm significantly improves model performance while exhibiting strong generalization capabilities.
Xiang Wang, Lei Tian, Jiahao Gan et al.· Neural Networks· 0 citations
In federated learning (FL), client data often suffer from the challenges of data distribution imbalance such as Non-IID and noisy labels. Crucially, these two issues are highly coupled and mutually exacerbating: Non-IID data complicates the identification of noisy labels, while noisy labels severely amplify local model drift. The compounding effect leads to severe training instability and degrades model convergence. To tackle these issues, we propose FedCRT, a Federated Centroid-based Robust Training framework designed to jointly mitigate both issues. At the client level, we introduce a centroid-based noisy label correction mechanism and class-conditional feature centroids. To prevent confirmation bias during this process, we devise a dual-confidence strategy for cautious and accurate label correction, accompanied by a robust loss function to maintain consistent optimization directions. At the server level, we develop a two-stage quality-aware aggregation strategy. It not only mitigates Non-IID-induced client drift but also dynamically assigns greater aggregation weights to clients with higher estimated clean data proportions. Extensive experiments have been conducted on benchmark datasets. The experiment results demonstrate that our method performs well in noisy label detection and correction on different clients under highly heterogeneous label noise scenarios which prove its effectiveness and robustness.
Xu-Ting He, Jiayi Huang, Jinbo Xiong et al.· Fall Joint Computer Conferen...· 0 citations
Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a federated learning framework that performs update selection in function space using an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client.
Mirko Konstantin, S. Zachow, Anirban Mukhopadhyay· 0 citations
This work proposes a data-Quality-aware aggregation framework by introducing an Evolutionary-computation-inspired de-sign into Federated learning ( FedEvoQ), with a lightweight dual-branch architecture.
A novel FL framework is presented, FedPhoenix, that stochastically re-sets partial parameters in each round to destroy some features of the global model, guiding FL training to learn multiple generalized features for inference rather than specific overfitting features.
Jiahao Wu, Ming Hu, Yanxin Yang et al.· Advances in Neural Informati...· 1 citation