TL;DR
A continual learning model that effectively leverages the extraction of GCF-based information, enabling it to alleviate catastrophic forgetting through cognitive condensation of GCF, and efficiently manages the rapid growth of stored data points in the storage space.
Experiments show that FiUni can effectively infer latent batch-level task affiliations and achieve competitive performance against advanced task-aware CL methods with fewer trainable parameters.
We explore catastrophic forgetting in the context of large pre-trained models. By considering forgetting as a geometric problem in the input space of each weight matrix, we uncover a natural retention objective under which updates produced by gradient-based optimizers are suboptimal. Following this observation, we prop...
This work proposes a dynamic width expansion method that increases the number of neurons within existing layers according to a normalized loss criterion, without requiring task-specific information.
This work identifies three sources of routing error (feature-level, task-level, and class-level misalignment) and proposes Feature Distribution Calibration (FDC), which improves final accuracy in all 40 method-dataset pairs by 4.39 percentage points on average.
Continual learning requires a model to retain knowledge of old tasks while sequentially learning new tasks, but standard neural networks typically suffer from catastrophic forgetting in this setting. To address this challenge, a new method is proposed based on a learnable knowledge transfer network. Specifically, a tra...
Together, these results show that composing complementary mechanisms substantially improves long-horizon memorization beyond what any individual mechanism achieves.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
