Skip to content
Log in
Review

Continual Learning in Transition

Zhi-Yan Hou Dan Zhang Tao Feng Liyuan Wang Wei Li Xiangzhao Hao Hongyan An Junfeng Fang Hao-Kai Ma Zhaohui Xu Xinyu Tang Haiyun Guo Jinqiao Wang Tat-Seng Chua
Aug 2026 · 0 citations · 177 references
Computer Science

TL;DR

Anchored by this tri-axial framework, representative methods are systematically surveyed, the ongoing transition of continual learning is traced, and the key challenges, broader implications, and future directions arising from this paradigm shift are discussed.

Abstract

Classical continual learning (CL) has primarily focused on enabling models to update and retain knowledge through parameter-centric mechanisms, e.g., training strategies, architectural designs, and weight adaptation. However, emerging paradigms are reshaping the scope of CL beyond this traditional model adaptation view. For instance, on-policy learning broadens the space of update mechanisms; test-time training extends CL from the training phase to inference; and external harness components such as memory, skill libraries, and interaction protocols extend the evolutionary boundaries of model capabilities far beyond the static parameter space. Collectively, these developments indicate a transition from parameter-centric learning toward system-level adaptation. To characterize this transition, we examine the evolution of continual learning through three dimensions: When, How, and Where learning occurs. The How dimension encompasses off-policy, on-policy, and beyond-gradient optimization mechanics. The When dimension captures evolution across pre-training, post-training, and inference-time stages. The Where dimension delineates updates occurring within internal parameters versus external structural constraints. Anchored by this tri-axial framework, we systematically survey representative methods, trace the ongoing transition of continual learning, and discuss the key challenges, broader implications, and future directions arising from this paradigm shift.

arXiv Semantic Scholar
View source

Similar papers

Preprint Jul 2026

To Retain or to Adapt? Generalizing Continual Learning

This work formalizes CL as an online optimization problem governed by the interaction between environmental and learning dynamics, and introduces Transfer Efficiency as a quantitative measure of the tension between Instability, the bias inherited from conflicting past experience, and Transient Error, the optimization cost of learning new tasks from scratch.

Giulia Lanzillotta, Mandana Samiei, D. Precup et al. · 0 citations
Preprint Jul 2026

When Does Continual Learning Require Learning

The results suggest that continual learning is not a single capability: different patterns of environmental change require fundamentally different update behaviors, determining when adaptation must be learned inside model weights and when it can be achieved through external scaffolding.

A. Harrington, Nayan Saxena, Michael Murphy et al. · 1 citation · ⚡1
Review Open access Aug 2026

Modern Continual Learning with Foundation Models, Evaluation Challenges, and Future Directions

This review provides a structured overview of major CL paradigms, including task-incremental, domain-incremental, class-incremental, online, multimodal, and federated CL, and examines the theoretical foundations of CL, particularly the stability–plasticity dilemma, catastrophic forgetting, transfer dynamics, and representation learning.

Zahid Ullah, Minki Hong, Jihie Kim · 0 citations
Preprint Aug 2026

Learning What to Remember and What to Internalize in LLM Self-Evolution via Adaptive Memory-Parameter Coordination

Large language model agents increasingly operate in dynamic environments where tool interfaces, APIs, and user requirements change after deployment. Existing self-evolution methods mainly follow two paradigms: harness-based approaches, which externalize feedback into editable memories or skills for rapid adaptation, and parameter-based approaches, which internalize experience into model parameters for deeper capability improvement. However, using either mechanism alone creates a trade-off between flexibility and performance. This paper asks how an agent can coordinate both channels to achieve robust self-evolution. We present COVE, a unified agent self-evolution framework that combines harness-based and parameter-based learning through task-aware routing, stage-aware scheduling, and knowledge optimization. Through this design, COVE treats self-evolution not as indiscriminate accumulation of experience, but as a coordinated process that matches tasks and knowledge types to appropriate learning mechanisms. Experiments across multiple task categories show that COVE outperforms single-channel evolution strategies, demonstrating more robust and efficient improvement under changing environments.

T. Ji, Zhenya Huang, Jiayu Liu et al. · 0 citations
Review Open access Aug 2026

Reinforcement Learning in the Era of Large Language Models: Challenges and Opportunities

A systematic literature review on how RL are adapted and scaled as a fundamental post-training tools and how innovations in the RL pipeline enhance the domain-specific LLMs is conducted.

Qianyue Hao, Lin Chen, Xiaoqian Qi et al. · 1 citation