Deep neural networks excel in various tasks but struggle to generalize across evolving data distributions, leading to significant performance degradation under domain shifts. Domain incremental learning (DIL) addresses this challenge by enabling models to continuously adapt while retaining prior knowledge. Among existing DIL approaches, the parameter-isolation paradigm achieves state-of-the-art performance. However, these methods often adopt a one-size-fits-all approach to adapt to new domains, resulting in either insufficient learning capacity or redundant parameters. In this work, we propose BPG, a unified framework that addresses both challenges through two complementary components: BPG-Adapter, which dynamically determines each domain's adapter hidden dimension based on domain-specific feature separability, and BPG-Inference, a soft domain mixture strategy that integrates multiple domain-specific models at test time, mitigating domain ID misselection. Experimental results on DomainNet, CDDB, and CORe50 demonstrate that BPG consistently outperforms uniform adapter-based approaches and hard domain selection strategies, achieving state-of-the-art average accuracy while reducing forgetting to as low as 0.22% on DomainNet.
Although SDG methods improve performance under highly domain distinguishable stylized shifts, they exhibit limited robustness to background, correlation and corruption shifts on larger datasets, and increased shape bias does not consistently yield enhanced OOD performance, thereby underscoring the need for further research into developing more resilient and generalizable models.
K. Imbulgoda, Ruwan Tennakoon, W. Chuah et al.· International Journal of Com...· 0 citations
RSIAT significantly outperforms state-of-the-art methods in both performance and parameter efficiency, achieving superior stability–plasticity trade-offs with minimal trainable parameters.
Jiarui Zhao, Libo Huang, Xiangqi Li et al.· 0 citations
This work compares three fusion paradigms by the artefacts they reuse and suggests that Merge when experts already exist and cheap fusion is paramount; Mix RL when training a unified model without experts, with domain proportions adjusted for cross-domain transfer; and MOPD when preserving domain-specific gains matters more than surpassing teachers or minimizing end-to-end cost.
Sicheng Wu, Kai Yang, Yuchen Cai et al.· 0 citations
Miles decouples the learnable modules with the pre-trained model, exploiting prior information from intermediate features of the backbone network to enable more flexible parameter expansion, and orchestrating an efficient expansion of the parameter space through guided optimization.
Kai Jiang, Zisong Lin, Hongyuan Zhang et al.· IEEE Transactions on Image P...· 0 citations
A data maturity-deployment scale framework is introduced, arguing that engineering AI should be understood not as a search for a single best model, but as an evolving intelligence strategy that co-adapts with system maturity, operational context, and deployment scale.
Jay Lee, Hanqi Su, Ruoxin Wang et al.· Machine Learning: Engineerin...· 0 citations
Modern machine learning pipelines increasingly rely on reusing pretrained and foundation models across downstream tasks. These pretrained models can differ not only in performance but also in how they can be used: some only provide black-box predictions, while others may permit white-box access to internal representations that can be probed or fine-tuned. When deployed to the target domain in the presence of distribution shift, no single strategy, including zero-shot application, fine-tuning, or directly training a target-specific model, is uniformly the best. In this work, we propose Frontier Learning, a framework that treats a library of candidate models spanning different training histories and access regimes as complementary sources of information rather than mutually exclusive alternatives. Frontier Learning constructs a unified target-domain feature by concatenating internal representations from white-box candidates as well as prediction outputs from black-box candidates, then fits a lightweight, regularized supervised learner on this concatenated representation using labeled target data. Because the resulting hypothesis class contains predictors obtained by zero-shot reuse, fine-tuning, and direct training as special cases, empirical risk minimization over the frontier learner is guaranteed to be no worse, on the training sample, than any individual baseline. We evaluate the framework in simulations spanning varying degrees of source-target compatibility and in two real-world distribution-shift settings: visual domain adaptation on DomainNet/VisDA and clinical mortality prediction across intensive care unit domains using MIMIC-IV-Notes. Across all settings, Frontier Learning matches or outperforms the strongest individual reuse strategy, with the largest gains arising precisely when no single baseline is reliable across the range of shift considered.
