InfRA-FL: Information-Driven Robust Federated Learning via Saturation-Aware Reinforcement Learning
TL;DR
Experiments show that InfRA-FL outperforms state-of-the-art baselines by 5–10% in accuracy while accelerating convergence, establishing that principled information-theoretic observation and robust reward design suffice to secure RL-driven federated learning against targeted poisoning.
Abstract
Client selection is a critical mechanism for ensuring robust convergence in Federated Learning (FL) systems, yet it remains vulnerable to Non-IID data distributions and Byzantine attacks. Deep Reinforcement Learning (DRL) has shown promise for automated client selection, yet existing methods suffer from three structural deficiencies: observation ambiguity, where scalar states cannot distinguish malicious updates from benign heterogeneity; reward fragility, whereby attackers exploit unbounded feedback to hijack policy updates; and risk blindness, as risk-neutral agents overlook the inherent variance in client contributions. To address these deficiencies, we propose InfRA-FL, a robust adaptive framework. A mutual-information-based state construction extracts high-utility features, resolving observation ambiguity. A Saturation-Aware Robust Reward (SARR) mechanism applies soft-clipping to bound each client’s influence on the policy gradient, provably neutralizing reward poisoning. URA-PPO, an uncertainty-aware algorithm with a dual-head Critic, optimizes a risk-penalized objective that shifts the agent from risk-neutral to risk-averse decision-making. Experiments on MNIST and CIFAR-10 under 20% Byzantine adversaries show that InfRA-FL outperforms state-of-the-art baselines by 5–10% in accuracy while accelerating convergence, establishing that principled information-theoretic observation and robust reward design suffice to secure RL-driven federated learning against targeted poisoning.