This work addresses leakage through a learned obfuscate-and-recover scheme that protects participants' private datasets while still allowing an independently deployable model to be trained on the server side, making split-based federated LLM fine-tuning practically viable.
Abstract
Fine-tuning large language models (LLMs) on domain-specific data is essential for downstream adaptation. In many deployments, a participant cannot hold the complete model locally. This happens because the model owner keeps the full model proprietary, or because the participant lacks sufficient compute resources. Split Learning (SL) addresses this by partitioning the model between the participant and a server so that only a small portion runs locally. When the underlying data is additionally distributed across multiple institutions with privacy requirements, Federated Learning (FL) further enables collaborative training across participants by sharing only model updates instead of raw data. In this combined setting, each client transmits intermediate activations to the server, and for LLM fine-tuning, this exchange poses an inherent privacy paradox. The autoregressive nature of LLMs causes the transmitted activations to leak the input, and existing perturbation-based defenses are fundamentally ineffective in this setting. We address this leakage through a learned obfuscate-and-recover scheme that protects participants'private datasets while still allowing an independently deployable model to be trained on the server side. Experiments demonstrate that our approach achieves strong privacy protection with modest utility loss and system overhead, making split-based federated LLM fine-tuning practically viable.
i-FedLoRA provides privacy guarantees, improves model accuracy by up to 3.8%, and expedites training by 1.37-2.23×, and facilitates heterogeneous LoRA aggregation that selectively prioritizes high-confidence knowledge to filter DP-induced noise, thereby achieving robust knowledge transfer.
Nan Yan, Yu-Qing Li, Xiong Wang et al.· Proceedings of the 32nd ACM...· 0 citations
This work identifies and characterize the Depth-Performance Dilemma: the regime that maximizes system efficiency is precisely where fine-tuning quality collapses, and challenges the prevailing assumption that partition depth is a utility-neutral tuning knob and provide a structural foundation for stable distributed LLM...
Hariharan Ramesh, S. Murugaiyan, Jyotikrishna Dass· 0 citations
Client-Resolved Generation (CRG), a genera- tion interface that separates server-side generation from the lexical realization of input-derived content, is introduced, which provides a practical interface for privacy-sensitive cloud LLMs by reducing plaintext exposure across both input and output pathways while preserv-...
Jeongho Yoon, Chanhee Park, Yong-Chan Chun et al.· arXiv.org· 0 citations
RetFL is proposed, a CKKS-enabled robust aggregation framework for DFL that establishes a decentralized training workflow with VRF-based candidate selection and view change, and designs a weighted aggregation scheme that incorporates cosine similarity and a dynamic reputation mechanism to weight updates and suppress pe...
Yi-Cheng Huang, Zhou Zhou, You-Liang Tian et al.· Journal of King Saud Univers...· 0 citations
Cloud-based Large Language Model (LLM) inference services typically require users to submit plain-text inputs, thereby posing severe privacy risks. Existing privacy-preserving paradigms are mostly task-specific and often necessitate pervasive modifications to the entire server-side model. This reliance introduces subst...
Wentao Zhong, Yu-Ting Li, Di-Cong Yu et al.· IEEE Transactions on Informa...· 0 citations
Related blog posts
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 9, 2026
Assistant Professor Benjamin Lindquist studies how engineers deliberately injected randomness into early AI, giving rise to deep-learning models that sometimes hallucinate.