TAILS resolves cross-task ambiguity at the representation level, while leaving the original PTM, method-specific modules, and classifier unchanged, and can improve classification and task-inference performance with modest parameter overhead and negligible inference cost.
Abstract
Pre-trained models (PTMs) provide a strong foundation for continual learning by offering stable representations that facilitate lightweight adaptation to new tasks. However, adapting well to each task does not ensure reliable inference over all learned tasks. Since task boundaries are often artificial and semantically entangled, an input from an unknown task can remain ambiguous even with strong PTM features, making cross-task prediction a key bottleneck. We propose Task-Anchored Inference Latent Shaping (TAILS), a lightweight post-PTM module that can be integrated into diverse continual learners and optimized through a decoupled step. TAILS uses fixed task anchors as persistent references to accumulated knowledge. It interprets each sample's feature representation relative to these references, then composes relevant evidence across tasks into latent recall. Rather than selecting a task-specific path or adjusting classifier outputs, TAILS uses latent recall to directly correct the feature representation before prediction. It therefore resolves cross-task ambiguity at the representation level, while leaving the original PTM, method-specific modules, and classifier unchanged. Extensive experiments across multiple PTM-based continual learning paradigms show that TAILS can improve classification and task-inference performance with modest parameter overhead and negligible inference cost.
A systematic evaluation of representative task-adaptation methods shows that task adaptation is not merely a capability-improving step, but an alignment intervention in its own right, motivating multi-dimensional alignment evaluation as a standard component of post-training pipelines.
James Elcock, William F. Shen, Xinchi Qiu et al.· 0 citations
Continual Learning (CL) for Large Language Models (LLMs) faces a fundamental Stability-Plasticity Dilemma : balancing the plasticity to acquire new capabilities with the stability to preserve prior knowledge. While Parameter-Efficient Fine-Tuning methods, such as LoRA, enable efficient adaptation, we identify a critical flaw in current approaches termed Rank-Blindness : the enforcement of a single rank constraint across diverse tasks, which entangles task-shared and task-specific knowledge, leading to catastrophic forgetting of earlier tasks and underfitting on complex new ones. To address this, we propose S PA RTA, a novel rehearsal-free framework guided by a rank-spectrum perspective that explicitly dis-entangles knowledge into two orthogonal sub-spaces. Specifically, S PA RTA employs a low-rank branch to capture task-shared representations and a high-rank branch to model task-specific features. To integrate these complementary representations, we introduce a context-aware dynamic router that adaptively fuses the two branches based on input semantics, while an explicit orthogonality constraint minimizes interference between shared and specific parameter subspaces. This design effectively isolates task-specific updates from shared knowledge, preventing the overwriting of prior capabilities while preserving strong adaptation capacity. Extensive experiments demonstrate that S PA RTA achieves a superior stability-plasticity balance compared to single-rank baselines. Notably, the proposed spectral disentanglement strategy substantially reduces inter-task interference and yields strong zero-shot generalization on unseen tasks. Our code will be available at https://github. com/Xnhyacinth/SpaRTA .
Huanxuan Liao, Shizhu He, Yupu Hao et al.· Annual Meeting of the Associ...· 1 citation· ⚡1
RSIAT significantly outperforms state-of-the-art methods in both performance and parameter efficiency, achieving superior stability–plasticity trade-offs with minimal trainable parameters.
Jiarui Zhao, Libo Huang, Xiangqi Li et al.· 0 citations
This work observes that pooled token embeddings from a frozen LLM embedding layer already separate task distributions throughout the learning sequence, and concludes that a Gaussian mixture model fitted on these embeddings, without any gradient-based training, is sufficient for task-agnostic adapter selection at test time, eliminating the need for a learned gating module.
The Decoupled Representation Dynamic Network is proposed, which addresses two intertwined challenges: cross-task confusion from sequential training on predominantly current-task data, which biases decision boundaries toward recent tasks; and under-optimized shared representations in the backbone that cap long-term discriminability as tasks accumulate.
Bin Huang, Yifu Chen, Zhilin Wang et al.· 0 citations
To prevent the text-to-motion skill from exploiting shortcut pathways instead of learning language semantics, motion intuition distillation is introduced to ground text-to-motion generation in language semantics and a task-guidance module that dynamically adjusts actions based on high-level language instructions is introduced.
Yuehan Zhang, Ming Gong, Linpu He et al.· 0 citations