Experimental results of the FedAvg algorithm applied to the Human Activity Recognition domain show that, although FedAvg confirms a higher degree of personalization capabilities while keeping a high degree of generalization with respect to the traditional centralized learning, this result is not so obvious under stressful conditions, such as when varying class distribution over clients.
Abstract
The federated learning (FL) paradigm fosters distributed pervasive computing combined with artificial intelligence techniques, allowing for optimized data usage and improved mitigation of privacy concerns. Indeed, model training occurs on the client's local devices, and model parameters are subsequently shared with a centralized server. However, there is a need to find a tradeoff between models'personalization and generalization capabilities. In this paper, we design and implement several testing scenarios devoted to evaluating and comparing the centralized, local, and federated paradigm performances. We also design and implement a scenario that emulates a change in clients'data. We then present experimental results of the FedAvg algorithm applied to the Human Activity Recognition (HAR) domain to understand the trade-off between personalized and generalized accuracy. Results show that, although FedAvg confirms a higher degree of personalization capabilities while keeping a high degree of generalization with respect to the traditional centralized learning, this result is not so obvious under stressful conditions, such as when varying class distribution over clients.
Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. This dissertation presents methodologies for building efficient personalized models by identifying which strategies are effective in the global training stage and by showing how to preserve global knowledge while securing user-specific performance during local adaptation. First, we show that as data heterogeneity increases, the collapse of feature vectors is a more fundamental bottleneck than classifier weights, and propose a method that directly mitigates the discrepancy in representation magnitude between local and global models. Second, we analyze that a training approach that strengthens local alignment can induce forgetting of global knowledge (e.g., categories not observed locally), and propose a method that achieves both local alignment and global knowledge preservation by combining feature distillation based on the global model's feature vectors. Third, in federated personalized reward model learning with preference heterogeneity, we empirically verify the conventional belief that"increasing the number of global models yields better initialization,"and we show that when sufficient local fine-tuning is allowed, a single global initialization can instead provide stronger personalization performance. This study redefines the role of global initialization under data and preference heterogeneity and provides practical training strategies that simultaneously satisfy global knowledge preservation and personalization.
The fast growth of big data, along with the growing stricter privacy laws, poses tremendous challenges to the traditional centralized implementation of AI models. Federated Learning (FL) offers a decentralized solution; however, it is undermined by a number of performance limitations such as large communicational overhead, statistical heterogeneity due to nonindependent and identically distributed data, and small computational capability of edge clients. This paper presents an optimized AI-based FL architecture, which consolidates a client selection algorithm (that is based on reinforcement-learning) and a dynamic and fairness-conscious aggregation protocol, as well as a new query-fragment caching solution. The combined model is strategically focusing on high utility players, reducing skew in data during model aggregation, and lessening on-device unnecessary processing. The empirical analysis of CIFAR-10 and FEMNIST shows that the presented framework reduces the communication rounds by 38% and the model accuracy is increased by 5.7% when compared with Fed Avg and Fed Prox, at the same time, data privacy is also rigorously maintained.
Shashi Purna Kurapathi· International Conference Com...· 0 citations
This study implemented a comprehensive experimental framework for analysing FL performance using standard FL aggregation protocols FedAvg, FedProx, and SCAFFOLD in conjunction with Differential Privacy mechanisms; specifically, the Gaussian noise mechanism with Rényi Differential Privacy (RDP) accountants.
Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a federated learning framework that performs update selection in function space using an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client.
Mirko Konstantin, S. Zachow, Anirban Mukhopadhyay· 0 citations
Nowadays, split federated learning (SFL) has emerged as an effective paradigm for enabling privacy-preserving collaborative intelligence across heterogeneous devices with limited computation. However, SFL incurs significant communication overhead in wireless networks due to the uplink transmission of high-dimensional smashed data, which degrades network efficiency. To mitigate the communication bottleneck, we propose a prototype-based SFL framework ProtoSFL. Specifically, each selected client computes local prototypes for observed classes and uploads them to the server. Based on the received prototypes, the server derives global prototypes and optimizes a weighted objective that combines classification loss with prototype alignment loss. The server then updates the model accordingly and returns personalized prototype gradients to the clients. Simulation results verify the effectiveness of ProtoSFL in reducing communication overhead, achieving a substantial reduction in uplink communication, while maintaining competitive testing accuracy under various heterogeneous data settings compared with SFL baselines.
A review of federated learning through a structured taxonomy that covers its core architectural paradigms, major learning types, model training approaches, and aggregation mechanisms, and analyzes the principal challenges confronting FL, including privacy and security risks, statistical and system heterogeneity, communication constraints, and global model divergence.
Mahdiyeh Velaei, Hosna Ghahramani, Ali Ghaffari et al.· Cluster Computing· 0 citations