Task Vector Descent: Learning from Non-IID Batches
A central challenge in continual learning is to acquire new knowledge without forgetting what the model has already learned. This challenge appears in language model training when training data comes from various domain-, user-, or task-specific distributions that are encountered unevenly over time. In such settings, s...
Anton Baumann, Jonas Hübotter, Zeynep Akata et al.
· 0 citations