This work formalizes CL as an online optimization problem governed by the interaction between environmental and learning dynamics, and introduces Transfer Efficiency as a quantitative measure of the tension between Instability, the bias inherited from conflicting past experience, and Transient Error, the optimization cost of learning new tasks from scratch.
Abstract
The Continual Learning (CL) literature has long been driven by the goal of mitigating catastrophic forgetting. This objective rests on a pervasive, often unstated assumption: that a lifelong learner should approximate the Joint-Task Learning (JTL) solution and retain all previously acquired knowledge. We challenge this retention-centered premise, arguing that in non-stationary environments prioritizing retention can impede real-time adaptation. Shifting the focus to the Average Lifelong Error (ALE), we formalize CL as an online optimization problem governed by the interaction between environmental and learning dynamics. We introduce Transfer Efficiency as a quantitative measure of the tension between Instability, the bias inherited from conflicting past experience, and Transient Error, the optimization cost of learning new tasks from scratch. Under mild convergence conditions, holding across linear and neural network models, this decomposition yields a Critical Task Duration: a closed-form threshold beyond which historical knowledge transitions from a warm-start advantage to an optimization liability whenever retention induces a positive stationary bias. We validate these theoretical predictions on continual image classification and reinforcement learning benchmarks. Finally, by connecting continual learning to the online learning framework of predictable sequences, we show that JTL is only one instance of a broader family of objectives, and we propose a new general class of continual learning algorithms, which we call Predictive Continual Learning. Predictive CL algorithms optimize expected future performance under an explicit, dynamically updated model of future tasks. As a proof of concept, we analyze a Window algorithm that interpolates between JTL and Independent-Task Learning (ITL), outperforming both under controlled distributional drift.
Continual post-training is becoming a central paradigm for adapting vision-language models to evolving tasks. Recent work has increasingly favored reinforcement learning over supervised fine-tuning, driven by the belief that reinforcement learning is inherently less prone to forgetting. However, the belief remains insufficiently validated, as existing evidence is largely drawn from outdated or homogeneous benchmarks. We revisit this assumption under recent and diverse multimodal reasoning tasks. To this end, we introduce MRCL, a Multimodal Reasoning Continual Learning benchmark. Experiments on MRCL show that standard reinforcement learning still suffers from severe catastrophic forgetting during continual post-training. We trace this failure to an objective mismatch: the KL regularization used in common policy optimization methods is evaluated on current-task data, whereas forgetting is caused by behavioral drift on prior-task distributions. To address this problem, we propose Continual Policy Optimization (CPO), a replay-free framework grounded in a prior-task behavioral KL objective. CPO relaxes the intractable historical KL constraint into sparse parameter-movement regularization, limiting policy drift without storing old data. Extensive experiments across multiple model scales show that CPO consistently reduces forgetting while preserving, and in some cases improving, pretrained model capabilities. On Qwen3-VL-8B, CPO reduces forgetting by 13.7% and improves pretrained capability by 7.0%. The implementation code is available at https://github.com/MaolinLuo/CPO.
Mao-Lin Luo, Zhe-Xu Wang, Zi-Hao Zhou et al.· 0 citations
Anchored by this tri-axial framework, representative methods are systematically surveyed, the ongoing transition of continual learning is traced, and the key challenges, broader implications, and future directions arising from this paradigm shift are discussed.
Zhi-Yan Hou, Dan Zhang, Tao Feng et al.· 0 citations
The results suggest that continual learning is not a single capability: different patterns of environmental change require fundamentally different update behaviors, determining when adaptation must be learned inside model weights and when it can be achieved through external scaffolding.
A. Harrington, Nayan Saxena, Michael Murphy et al.· 1 citation· ⚡1
From this analysis, Interference-Gated Functional Allocation (IGFA) is derived, a replay-free, Fisher-free method that shares directions when tasks align and protects them when they conflict and achieves lossless retention when tasks are structurally separable.
The results suggest that the empirical success of memory replay goes beyond the mitigation of forgetting, actively reintroducing the benefits of data co-observation into the learning process.
Timm Hess, Abhishek Jha, Gido M. van de Ven et al.· 0 citations
Continual learning (CL) aims to incrementally update machine learning models from a stream of data without forgetting previously acquired knowledge. CL is highly relevant in many real-world applications such as manufacturing, where storing historical data for retraining is often infeasible due to volume, governance, or system constraints. Yet, a common challenge in continual learning (CL) is catastrophic forgetting, where the performance on old tasks drops after new, additional tasks are learned. In this paper, we propose a novel framework called ReCL to slow down forgetting in CL. Our framework exploits an implicit bias of gradient-based neural networks due to which these converge to margin maximization points. Such convergence points allow us to reconstruct old data from previous tasks, which we then combine with the current training data. Our framework is flexible and can be applied on top of existing, state-of-the-art CL methods. We first demonstrate the performance gain from our framework across a large series of experiments on three established public CL datasets (MNIST, CIFAR10, TinyImageNet), a public industrial dataset (SECOM), and across two different scenarios (class incremental and domain incremental learning). Then, we evaluate the performance of ReCL for predictive maintenance in a manufacturing environment at Bosch, a global engineering company, using an internal real-world time-series feature dataset captured from a high-volume precision-machining process. Lastly, we apply our framework to streaming machine data from real-world industrial data at Bosch. Across all our experiments, we find large performance gains through ReCL. To the best of our knowledge, our framework is the first to address catastrophic forgetting by leveraging models in CL as their own memory buffers.
Pascal Janetzky, T. Schlagenhauf, Michael Klar et al.· Proceedings of the 32nd ACM...· 0 citations