From this analysis, Interference-Gated Functional Allocation (IGFA) is derived, a replay-free, Fisher-free method that shares directions when tasks align and protects them when they conflict and achieves lossless retention when tasks are structurally separable.
Abstract
Continual learning commonly relies on post-hoc mechanisms such as replay, elastic regularization, or distillation. This work argues that forgetting should instead be modeled directly as interference between tasks. In the frozen-feature regime, forgetting from learning a new task is exactly the interference energy induced on the old task. In deep networks, the same quantity is recovered through path-averaged curvature with minimal additional forward passes. When task supports are disjoint, forgetting can be eliminated structurally and when task supports overlap in conflicting directions, a non-zero distortion floor is unavoidable. The same geometry optimally merges models through task-aware orthogonalization. From this analysis we derive Interference-Gated Functional Allocation (IGFA), a replay-free, Fisher-free method that shares directions when tasks align and protects them when they conflict. Across benchmarks, IGFA achieves lossless retention when tasks are structurally separable and moves unavoidable cost from irreversible forgetting into deferred but recoverable plasticity when they are not. It matches the strongest replay-free structural baselines on dissimilar-task streams and improves on unconditional projection when similarity makes transfer worth preserving.
This work formalizes CL as an online optimization problem governed by the interaction between environmental and learning dynamics, and introduces Transfer Efficiency as a quantitative measure of the tension between Instability, the bias inherited from conflicting past experience, and Transient Error, the optimization cost of learning new tasks from scratch.
Giulia Lanzillotta, Mandana Samiei, D. Precup et al.· 0 citations
AI Engram (Kwon et al., 2026) formalizes the four engram criteria of neuroscience as a constrained inverse problem in weight space and solves it closed-form: concept-specific memory traces become linear objects that can be extracted once and combined arithmetically. Appendix F states the Compositional Memory States Hypothesis: edited models live on"a commutative manifold where the integration of A and B reaches a consistent equilibrium regardless of the learning sequence."The evidence base is single and paired edits -- in materials terms, single-cycle tests, in which fatigue accumulation is structurally invisible. Whether the hypothesis holds under sequential load is exactly the"temporal dynamics"question the paper defers to future work. We run that test on the authors'own reference implementation, at their reported best edit strength (TOFU alpha=0.6, a choice favoring the linearity hypothesis), with pre-registered predictions, across three model charges (two vendors, two architecture families). Four findings replicate across all three: (1) zero-shot composition and sequential re-calibrated editing diverge by 61-71% of the edit magnitude; (2) cut order is not interchangeable, and the effect scales with concept overlap -- in one charge the order of cutting two Paris landmarks decides whether an uninvolved third concept survives; (3) the survivors'layer-input covariances -- the method's own sufficient statistics, read as strain gauges -- drift monotonically with every further cut, in every surviving concept, in every charge; (4) erased knowledge partially returns under subsequent unrelated cuts. Appendix F's commutative-manifold hypothesis is thereby falsified for sequential editing; the single-edit results of the original paper are untouched. For unlearning-as-compliance: erasure certified today does not certify the artifact after its next edit.
The results suggest that the empirical success of memory replay goes beyond the mitigation of forgetting, actively reintroducing the benefits of data co-observation into the learning process.
Timm Hess, Abhishek Jha, Gido M. van de Ven et al.· 0 citations
Existing Online Continual Learning (OCL) methods, particularly those based on Parameter-Efficient Fine-Tuning (PEFT), predominantly operate under a Static Inference Assumption, which means freezing model parameters immediately after training. This paradigm ignores the inevitable covariate shift in non-stationary test streams, resulting in suboptimal decision boundaries that fail to adapt to the immediate testing context. Furthermore, despite the computational efficiency of current analytic learning approaches, their reliance on Gaussian Random Projection introduces severe geometric distortion and feature aliasing in finite dimensions. Drawing inspiration from the Learning to Memorize at Test Time mechanism and the Pattern Separation process in the hippocampus, we propose Test-Time Memory Calibration (TTMC), a novel gradient-free analytic framework. Unlike traditional approaches that treat memory as a static statistic, TTMC views it as a dynamic associative pool. Specifically, we introduce a transductive calibration mechanism that seamlessly fuses the second-order statistics of the unlabelled test stream into the accumulated long-term memory via a closed-form solution, allowing for real-time alignment with the test distribution. To ensure the geometric stability of this dynamic test-time calibration, we theoretically prove the necessity of substituting random projections with Isometric Orthogonal Projections, which guarantee energy conservation and minimize feature interference. Our empirical evaluation demonstrates that TTMC consistently establishes a new state-of-the-art across diverse benchmarks. Notably, these performance gains are realized with zero backward passes and negligible computational overhead, underscoring the superior efficiency and structural robustness of our dynamic analytic framework. Code is available at https://github.com/hanyuyang99/TTMC.git.
Yuyang Han, Ziyu Li, Diwei Su et al.· Proceedings of the 32nd ACM...· 0 citations
Lifelong learning requires acquiring new knowledge without erasing the old. Yet neural population codes for familiar stimuli and behaviors change over days and weeks. This coexistence of stable memory and changing internal codes may depend on how a learning system prevents forgetting. We therefore tested whether different continual-learning mechanisms produce distinct patterns of representational drift. We trained convolutional networks on sequential image classification tasks and recurrent networks on sequences of cognitive tasks, tracking fixed probe representations across learning. Experience replay preserved earlier tasks in both architectures while representations drifted progressively with the number of intervening tasks. Drift was structured: later visual-processing stages and recurrent units'temporal tuning were especially labile, whereas coarse class organization and task-relevant temporal structure persisted. In contrast, algorithm that strongly anchored weights nearly froze representations. Directly anchoring an old representation during replay likewise suppressed drift and impaired acquisition of subsequent tasks. Together, these results link representational drift to the stability--plasticity trade-off: its magnitude is shaped by the mechanism that protects old knowledge, and suppressing it can restrict future learning. Drift may therefore provide an observable signature of the constraints that enable continual learning in brains and machines.