TL;DR
RSIAT significantly outperforms state-of-the-art methods in both performance and parameter efficiency, achieving superior stability–plasticity trade-offs with minimal trainable parameters.
Miles decouples the learnable modules with the pre-trained model, exploiting prior information from intermediate features of the backbone network to enable more flexible parameter expansion, and orchestrating an efficient expansion of the parameter space through guided optimization.
Deep neural networks excel in various tasks but struggle to generalize across evolving data distributions, leading to significant performance degradation under domain shifts. Domain incremental learning (DIL) addresses this challenge by enabling models to continuously adapt while retaining prior knowledge. Among existing DIL approaches, the parameter-isolation paradigm achieves state-of-the-art performance. However, these methods often adopt a one-size-fits-all approach to adapt to new domains, resulting in either insufficient learning capacity or redundant parameters. In this work, we propose BPG, a unified framework that addresses both challenges through two complementary components: BPG-Adapter, which dynamically determines each domain's adapter hidden dimension based on domain-specific feature separability, and BPG-Inference, a soft domain mixture strategy that integrates multiple domain-specific models at test time, mitigating domain ID misselection. Experimental results on DomainNet, CDDB, and CORe50 demonstrate that BPG consistently outperforms uniform adapter-based approaches and hard domain selection strategies, achieving state-of-the-art average accuracy while reducing forgetting to as low as 0.22% on DomainNet.
Exemplar-free class-incremental learning (EFCIL) poses the challenge that models cannot access data from previous tasks when learning new classes, leading to catastrophic forgetting. Recent methods freeze the feature extractor after the initial task and adapt only the classification mechanism to new classes, achieving strong performance. However, they largely overlook improving the frozen feature extractor's adaptability to unseen classes, which limits further performance gains. To address this limitation, we propose a novel complementary asymmetric representation learning (CARL) framework to enhance the model's adaptability to unseen classes. The core of CARL is the asymmetric representation learning (ARL) architecture, which combines a base encoder that extracts discriminative features for the initial task with a projection multilayer perceptron (MLP) head appended to its output. By introducing the projection head, the original output representation of the base encoder becomes an intermediate representation in the projected branch, encouraging the encoder to learn more generalizable features. This allows the model to preserve discriminative representations for the initially learned classes while improving its adaptability to unseen classes. In addition, the CARL framework employs two parallel ARL branches and aggregates their Mahalanobis distances during inference to improve prediction performance. To evaluate the efficacy of CARL, we compare it with recent methods using three widely recognized datasets. The proposed approach improves average accuracy over the best competing method by 3.70 percentage points on CIFAR-100, 2.63 percentage points on Tiny-ImageNet, and 2.90 percentage points on ImageNet-Subset.
Experimental findings show that pretrained models outperform those trained from scratch in terms of accuracy, convergence speed, and robustness, and a unified framework is proposed to integrate both processes in a deep learning pipeline.
This work observes that pooled token embeddings from a frozen LLM embedding layer already separate task distributions throughout the learning sequence, and concludes that a Gaussian mixture model fitted on these embeddings, without any gradient-based training, is sufficient for task-agnostic adapter selection at test time, eliminating the need for a learned gating module.
TAILS resolves cross-task ambiguity at the representation level, while leaving the original PTM, method-specific modules, and classifier unchanged, and can improve classification and task-inference performance with modest parameter overhead and negligible inference cost.