This work introduces a novel multi-domain federated learning framework in which lightweight client-side proxy models collaborate with a server-side Foundation Model (FM) to learn new concepts without sharing private data.
Abstract
Recent data protection laws have accelerated the adoption of Federated Learning (FL) for privacy-preserving decentralized training. Nevertheless, increasing model sizes impose substantial computational demands on client devices, limiting FL applicability in resource-constrained settings. We introduce a novel multi-domain federated learning framework in which lightweight client-side proxy models collaborate with a server-side Foundation Model (FM) to learn new concepts without sharing private data. Our approach, EFFEKT, enables efficient server-side training of domain-specific LoRA adapters while preserving feature-space alignment between the FM and proxy extractors via novel bi-directional cross-distillation strategies. Experiments on multiple real-world datasets and deployments on low-power edge devices demonstrate improvements over state-of-the-art baselines in most considered domains while maintaining lightweight computation at the client side.
A domain-aware proxy selection framework to better adopt proxy data for OOD problems is proposed and the experimental results show that the proposed models effectively address the challenges of distribution shifts under OOD with and without proxy data.
This work addresses leakage through a learned obfuscate-and-recover scheme that protects participants' private datasets while still allowing an independently deployable model to be trained on the server side, making split-based federated LLM fine-tuning practically viable.
Heng Jin, Chao-Yu Zhang, He-Xuan Yu et al.· 1 citation
FedSLM, a parameter-centric framework for federated fine-tuning with heterogeneous compressed clients, is proposed, which provides theoretical guarantees for adapter-level aggregation, subspace-alignment bounds for cross-group fusion, and a characterization of how the confidence loss mitigates weak-supervision noise.
A novel FL framework is presented, FedPhoenix, that stochastically re-sets partial parameters in each round to destroy some features of the global model, guiding FL training to learn multiple generalized features for inference rather than specific overfitting features.
Jia-Hao Wu, Ming Hu, Yanxin Yang et al.· Neural Information Processin...· 1 citation
In FL-OA, the server collaborates with third-party organization that holds an additional root dataset to perform outsourced auditing, thereby enabling the server to achieve robust aggregation without strong assumptions, demonstrating that FL-OA outperforms existing defense methods against Byzantine attacks.
Hongliang Zhang, Zhongyuan Yu, Fenghua Xu et al.· 0 citations
A review of federated learning through a structured taxonomy that covers its core architectural paradigms, major learning types, model training approaches, and aggregation mechanisms, and analyzes the principal challenges confronting FL, including privacy and security risks, statistical and system heterogeneity, commun...
Mahdiyeh Velaei, Hosna Ghahramani, Ali Ghaffari et al.· Cluster Computing· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
