Skip to content

FedDPRL: Communication-Efficient Differentially Private Federated Reinforcement Learning With Adaptive Gradient Compression

2026 · IEEE Transactions on Communications · Vol 74, pp. 13899-13912 · 0 citations · 55 references

Abstract

Cross-silo federated reinforcement learning (FRL) trains a shared policy across silos that must simultaneously respect a per-round uplink budget and on-device privacy. Although differential privacy (DP) and gradient compression each have mature solutions in supervised federated learning, naively stacking them in the policy-optimisation regime fails, for two structural reasons. First, whole-client DP injects noise whose ratio to the signal scales as <inline-formula> <tex-math notation="LaTeX">$\sqrt {d}/N$ </tex-math></inline-formula>, which for neural policies exceeds one at every practical <inline-formula> <tex-math notation="LaTeX">$\varepsilon $ </tex-math></inline-formula>, so the policy collapses to random. Second, a single shared compression mask cannot represent each client’s descent direction under non-i.i.d. environments, driving the compressor’s contraction constant toward zero and destroying learning. In this paper, we propose FedDPRL, a federated policy-optimisation algorithm built from two coupled designs. Specifically, a record-level (per-trajectory) Gaussian mechanism clips and noises each trajectory, lifting the effective sample size to <inline-formula> <tex-math notation="LaTeX">$R=NB$ </tex-math></inline-formula> and removing the <inline-formula> <tex-math notation="LaTeX">$\sqrt {d}/N$ </tex-math></inline-formula> barrier so that DP becomes viable on neural policies. Additionally, an error-feedback Top-<inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> compressor with per-client masks preserves each client’s own descent direction while cutting the uplink by an order of magnitude. Experimental results across classic control, continuous control (MuJoCo), and vision with <inline-formula> <tex-math notation="LaTeX">$N=20$ </tex-math></inline-formula> clients show that FedDPRL matches dense DP-FedAvg-RL utility at 13–<inline-formula> <tex-math notation="LaTeX">$14\times $ </tex-math></inline-formula> less uplink, while a shared mask collapses to near-random.

View source

Similar papers

#artificial intelligence Preprint Aug 2026

Provably Efficient Federated Reinforcement Learning with Linear Function Approximation and Logarithmic Communication Cost

Fed-LSVI is proposed, the first provably efficient federated algorithm for online reinforcement learning with linear function approximation in episodic Markov decision processes and achieves a regret bound of $\widetilde{\mathcal O}(\sqrt{Md^3H^4T})$, matching the best-known regret for multi-agent online reinforcement...

Zi-Han Liang, Haochen Zhang, Ling-Zhou Xue · 0 citations

FedeRage: Provably Convergent Agnostic Federated Learning under General Client Drift

Federated learning (FL) enables collaborative model training without sharing raw data, but its performance degrades under non-IID data and stochastic client participation. Remedies built on classical Federated Averaging (FedAvg) typically presuppose that client participation probabilities are known to the server, which...

Herlock Rahimi, Dionysis Kalogerias · 0 citations
#machine learning Preprint Sep 2026

Personalized Federated Reinforcement Learning via Model-Agnostic Meta-Learning: Convergence of Exact and Hessian-Free Meta-Policy Gradients

Together these results identify the adaptation step size as a tunable personalization knob and the curvature estimate as the quantity that governs whether exact meta-gradients are affordable.

Ali Beikmohammadi, Sarit Khirirat, Sindri Magnússon · 0 citations
#edge computing Open access Aug 2026

FedMARL-LTI: Federated Multi-Agent Reinforcement Learning with LLM-Compatible Threat Intelligence for Cooperative Cyber Defense

FedMARL-LTI is presented, a federated multi-agent reinforcement learning framework whose architecture answers both pressures with a single decision: each organization’s threat intelligence is shared only as a differentially private 768-dimensional semantic embedding, never as raw data.

Fatih Şahin · 0 citations
Conference Aug 2026

Federated Learning Architectures and Communication-Efficient Optimisation for Privacy-Preserving Distributed AI Systems

Federated learning (FL) trains a shared model across data holders that cannot pool their records, but deployments remain bounded by three coupled costs: uplink traffic from repeated model exchange, accuracy loss under statistically heterogeneous clients, and the information that updates still leak. These are usually at...

Harshavardhan Peddireddy, Sandeep Kumar Gadde, Prasad Bheemavarapu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.