2025· Neural Information Processing Systems· pp. 115103-115133· 1 citation· 40 references
Computer Science
TL;DR
This work investigates memorization in transformer-based language models by analyzing their memorization dynamics during training over multiple epochs and finds that memorization is neither a constant accumulation of sequences nor simply dictated by the recency of exposure to these sequences.
Abstract
Memorization in language models is a critical yet poorly understood phenomenon. In this work, we investigate memorization in transformer-based language models by analyzing their memorization dynamics during training over multiple epochs. We find that memorization is neither a constant accumulation of sequences nor simply dictated by the recency of exposure to these sequences. Instead, much like generalization, memorization appears to be driven by pattern recognition. Tracking memorization dynamics in mixed datasets, we observe that models memorize different sub-datasets in distinct bursts, suggesting that each subset is associated with unique underlying patterns, and that the model prefers to learn these patterns in a consistent order. We also find that easily learnable patterns tend to support generalization on unseen data, while more complex patterns do not. Furthermore, in datasets with weak or absent patterns, larger models may delay memorization relative to smaller ones, a behavior we term overthinking . Our results show that the subset of sequences memorized by a model over time is not arbitrary, and give insights into the internal processes a model goes through during training. Our code is available at: https://github.com/mdrpanwar/memorization-patterns .
It is shown that matched comparisons enable rigorous, calibrated memorization claims, and reveal where prior setups have validity issues, which require a valid memorization claim and near-certain generation within a realistic budget.
A. F. Cooper, Marika Swanberg, Jamie Hayes et al.· 0 citations
Self-patching identifies activation locations where relocating representations substantially improves failed generalization cases and design a simple heuristic strategy which recovers 58--75\% of the oracle headroom in generalization failure.
Diffusion models have attained remarkable success across a wide range of generative tasks. A key challenge lies in understanding the mechanisms that prevent their memorization of training data and allow generalization. In this work, we investigate the role of the training dynamics in the transition from generalization to memorization. Through extensive experiments and theoretical analysis, we identify two distinct timescales: an early time τgen at which models begin to generate high-quality samples and a later time τmem beyond which memorization emerges. Crucially, we found that τmem increases linearly with the training set size n, whereas τgen remains constant. This creates a growing window of training times with n where models generalize effectively, despite showing strong memorization if training continues beyond it. It is only when n becomes larger than a model-dependent threshold that overfitting disappears at infinite training times. These findings reveal a form of implicit dynamical regularization in the training dynamics, which allows to avoid memorization even in highly overparameterized settings. Our findings are supported by numerical experiments with standard U-Net architectures on realistic and synthetic datasets, alongside a theoretical analysis using a tractable random features model studied in the high-dimensional limit.44 https://github.com/tbonnair/Why-Diffusion-Models-Don-t-Memorize. https://github.com/tbonnair/Why-Diffusion-Models-Don-t-Memorize.
Tony Bonnaire, Raphaël Urfin, Giulio Biroli et al.· Journal of Statistical Mecha...· 2 citations
Recent work has proposed that reasoning and memorization in language models can be characterized by a single representation direction, including methods that keep this direction fixed during reinforcement learning. We test two assumptions behind this view. First, are reasoning-oriented and factual-recall task groups approximately single-direction separable? Second, does the resulting geometry remain stable after GRPO? Using Qwen3-0.6B and a controlled 400-example dataset, we find that a one-dimensional projection can match a full 1024-dimensional linear probe with AUROC = 1.00 on the studied task groups. However, after GRPO, the corresponding direction is substantially reorganized: mean-direction cosine averages 0.453, probe-direction cosine 0.445, while direct representation drift reaches 0.511 at the final layer. Probe AUROC nevertheless remains 1.00. The evidence therefore supports single-direction decodability for the studied task groups but challenges fixed-direction stability: the information persists while its geometric realization changes.
This work proposes "Memorization-guided Data Reuse", a training paradigm that adaptively determines when and how data should be reused, enabling principled decisions on the number of training epochs and the scheduling of data replays.
Jingwei Zuo, Cong Zeng, Ilyas Chahed et al.· 0 citations
Analysis of the neural correlates of minutes-scale memorability using intracranial EEG recordings from nine patients with epilepsy performing a musical memory task reveals complementary neural mechanisms through which statistical structure shapes musical sequence memory, clarifying how the brain encodes complex, ecologically valid stimuli.
Mathieu Pham Van Cang, Paul Robert, Manuel R Mercier et al.· bioRxiv· 0 citations