How Learning Governs Unlearning across the Memorization-Generalization Spectrum
This paper first classify memorization- and generalization-heavy models using grokking in modular addition and compare their responses to unlearning, showing that the latter suffer greater retain damage, i.e., a larger performance drop on the retain set.