This work casts FCL as a stochastic control problem and proposes Federated Queue-regulated Continual Learning (FedQCL), a framework based on Lyapunov drift-plus-penalty (DPP) optimization that outperforms state-of-the-art baselines with respect to accuracy while significantly reducing forgetting under heterogeneous data distributions.
Abstract
Federated Continual Learning (FCL) is fundamental to real-world distributed learning systems, requiring models to adapt to sequential, non-IID data across clients while mitigating catastrophic forgetting and client drift. Existing approaches formulate continual learning (CL) as a sequence of per-task optimization problems, applied locally at each client and coupled through aggregation, using heuristic mechanisms such as replay, regularization, or projection-based constraints. However, forgetting in FCL is inherently a long-term, distributed phenomenon, arising from the interaction of temporal task evolution and cross-client heterogeneity, which is not explicitly regulated. In this work, we cast FCL as a stochastic control problem and propose Federated Queue-regulated Continual Learning (FedQCL), a framework based on Lyapunov drift-plus-penalty (DPP) optimization. FedQCL introduces virtual queues to track the accumulation of forgetting across tasks and clients, enabling explicit control of the stability-plasticity trade-off. By optimizing a DPP objective, the method jointly improves current-task performance while the queue-based formulation provides an interpretable and tunable mechanism to balance adaptation and retention through a single parameter, without requiring gradient projection or additional communication overhead. Empirical evaluations on standard benchmarks, including Split-CIFAR-10, Split-CIFAR-100, and Split-TinyImageNet, demonstrate that FedQCL outperforms state-of-the-art baselines with respect to accuracy while significantly reducing forgetting under heterogeneous data distributions.
Deploying machine learning surrogates in scientific simulations faces multifaceted challenges, primary among which is the lack of Continual Learning (CL) capabilities—specifically, the inability to adapt to new physical regimes without significantly degrading performance on prior ones. This is particularly problematic for autoregressive surrogates of time-dependent Partial Differential Equations (PDEs), where small prediction errors can accumulate over long rollouts and new physical regimes overwrite previously learned dynamics. We formulate this adaptation as a CL problem, demonstrating that while standard Experience Replay (ER) is a robust baseline across Advection-Diffusion, Burgers’, and Navier-Stokes equations, storing full high-resolution rollouts can be memory-inefficient. To address this, we introduce Replay-TS, a temporal-slicing replay strategy that stores compact autoregressive windows sampled across past simulations. Through empirical analysis, we show that Replay-TS exploits the low-frequency spectral redundancy of physical systems to enable sparse supervision for rollout steps. By preserving the contiguous historical context and sparsely penalizing the autoregressive target steps, Replay-TS improves retention performance under a fixed memory budget by leveraging higher sample diversity. Replay-TS consistently outperforms standard ER methods across standard 1D and 2D streams, achieving over a 30% MSE reduction in a mixed-physics stream, while remaining architecture-agnostic.
Hamed Hemati, Binh Duong Nguyen, Stefan Sandfeld· Machine Learning for Computa...· 0 citations
Results indicate that multi-factor reliability assessment and heterogeneity-aware adaptive forgetting improve the balance between useful knowledge retention and harmful-knowledge suppression in heterogeneous FCL.
A forecast-free reinforcement learning (RL) framework for DERA allocation that learns optimal policies directly from operational data, which preserves the interpretability and constraint satisfaction of DER model while adapting to stochastic demand variations through data-driven updates.
Abed AlRahman Al Makdah, Aravind Ramana, Shaofeng Zou et al.· 0 citations
This work formalizes CL as an online optimization problem governed by the interaction between environmental and learning dynamics, and introduces Transfer Efficiency as a quantitative measure of the tension between Instability, the bias inherited from conflicting past experience, and Transient Error, the optimization cost of learning new tasks from scratch.
Giulia Lanzillotta, Mandana Samiei, D. Precup et al.· 0 citations
Local SGD (FedAvg) reduces federated communication by running multiple local steps between server aggregations, but it suffers from “client drift” under non-IID data. Methods such as SCAFFOLD mitigate this drift by transmitting control variates, which roughly doubles the per-round payload. We propose Momentum-Buffered Local SGD (MB-LSGD), an algorithm that corrects client drift with no additional payload: each client maintains a local drift buffer updated from the standard model aggregate, and a half-step bias present in naive drift estimators is eliminated by an additive update rule. For smooth non-convex objectives, MB-LSGD attains an \(\mathcal {O}(1/\sqrt {M K R})\) convergence rate, matching centralized mini-batch SGD up to constants. The bound explicitly couples the buffer rate α with the participation rate p, providing concrete guidance for hyperparameter selection under partial client sampling. With 100 clients under Dir(0.1) partitioning, MB-LSGD reaches a peak accuracy of \(48.37\%\) on TinyImageNet, exceeding SCAFFOLD (\(47.49\%\)) at half the bandwidth, and a final accuracy of \(61.67\%\) on CIFAR-10, the highest among all stable baselines. MB-LSGD is also the only method whose final accuracy on CIFAR-10 matches its peak, indicating sustained convergence without late-stage degradation, and its halved per-round payload relative to SCAFFOLD translates to close to a 2 × reduction in end-to-end completion time in communication-bound deployments.
Aizierjiang Aiersilan· IEEE International Symposium...· 0 citations
We study federated online reinforcement learning with linear function approximation. While recent multi-agent reinforcement learning algorithms achieve strong regret guarantees, they typically require sharing raw trajectories. This reliance incurs a communication cost that scales linearly with the number of episodes and violates the privacy constraints of federated settings. To address these limitations, we propose Fed-LSVI, the first provably efficient federated algorithm for online reinforcement learning with linear function approximation in episodic Markov decision processes. By integrating a determinant-based event-triggered synchronization with a stepwise backward update mechanism, Fed-LSVI enables agents to collaboratively learn an optimal policy by exchanging only compressed sufficient statistics. We prove that Fed-LSVI achieves a regret bound of $\widetilde{\mathcal O}(\sqrt{Md^3H^4T})$, where $d$ is the feature dimension, $H$ is the horizon length, $M$ is the number of agents, and $T$ is the number of episodes per agent, matching the best-known regret for multi-agent online reinforcement learning with linear function approximation. Moreover, by following the stringent communication and privacy constraints of the federated setting, Fed-LSVI reduces the communication cost to only logarithmic dependence on $T$, representing a significant improvement over prior methods.