This paper proposes a new domain-specific parameter-isolation architecture that retains all past domains, and mitigates catastrophic forgetting through a full-order recurrent update, constructing a new expert using domain-specific data conditioned on all previously frozen models.
Abstract
To successfully deploy a model in time-varying environments such as streaming data prediction and sensing control, domain-incremental learning (DIL) has attracted attention since it aims to adapt a previously trained model to newly arriving domains, while reserving knowledge from earlier domains without accessing their data. Incremental learning across domains can be regarded as a recurrent update, in which the current model is obtained by updating the model carried over from previous domains. Conventional DIL approaches that rely on domain-invariant feature learning and weight regularization gradually overwrite or constrain parameters learned in previous domains, leading to catastrophic forgetting. Instead, this paper proposes a new domain-specific parameter-isolation architecture that retains all past domains. The proposed architecture mitigates catastrophic forgetting through a full-order recurrent update, constructing a new expert using domain-specific data conditioned on all previously frozen models. To achieve this, we incorporate data-free generative replay to reconstruct previous-domain data and cross-domain feature generation to recover later expert features missing from earlier domain samples. Finally, we apply the proposed model architecture to domain-agnostic incremental learning for audio classification, as defined in the DCASE 2026 Challenge Task 7. Consequently, we achieve micro and macro accuracies of 78.4% and 78.9%, respectively, representing increases of 33 and 25 percentage points over the Challenge baseline. Ablation studies are conducted to examine the effectiveness of each processing component in terms of classification accuracy.
We propose a domain-incremental learning framework for generative speech enhancement (SE) that learns from a sequence of datasets or domains recorded under diverse acoustic conditions. Fine-tuning a pretrained model on continuously evolving domains leads to catastrophic forgetting of previously acquired knowledge, whil...
Manjunath Mulimani, Annamaria Mesaros, Minje Kim et al.· 0 citations
Deep neural networks excel in various tasks but struggle to generalize across evolving data distributions, leading to significant performance degradation under domain shifts. Domain incremental learning (DIL) addresses this challenge by enabling models to continuously adapt while retaining prior knowledge. Among existi...
Qiang Wang, Songlin Dong, Shao-Kun Wang et al.· 0 citations
This work proposes Local Support Learning (LSL), a general-purpose framework that augments gradient-based training for retention of prior capabilities without access to prior data, and presents a gate based on a Gaussian Mixture Model, whose likelihood decays rapidly away from its training data, giving it a natural ten...
Assaf Ben-Kish, Akarsh Kumar, James R. Glass et al.· 0 citations
This study investigates exemplar-free analytic continual learning as a principled alternative, in which a linear classifier is updated in closed form without storing historical recordings or performing incremental back-propagation, and previously learned weights remain intact by construction.
Catastrophic forgetting is a major challenge for deep learning models when they are incrementally trained on a sequence of new data. Reducing this forgetting in image and video data has been the primary research focus, but less attention has been given to textual domains, where discrete token distributions and semantic...
Z. Nizamani, Mir Sajjad Hussain Talpur, P. K. Butt et al.· Electronics· 0 citations
Parameter-efficient fine-tuning (PEFT), particularly Low-Rank Adaptation (LoRA), is widely used to adapt large language models (LLMs) to specialized downstream domains. However, although the pretrained backbone remains frozen, a domain-adapted LoRA branch may interfere with the model’s original representations and pred...
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026