Preprint
Aug 2026
Layers Matter: Why Continual Learning Regularization Should Be Layer-Adaptive
Continual learning regularizers like EWC fight forgetting by penalizing changes from previous-task parameters with per-parameter importance, typically diagonal Fisher values. Per-parameter looks more flexible than per-layer, but each layer's diagonal Fisher is a weak summary of its actual curvature, missing the top-eig...
Brian B. Moser, Ahmed Anwar, T. Nauen et al.
· 0 citations