Skip to content

Early Learning Shapes Later Directions Of Representation Change In Continual Learning

Sep 2026 · 0 citations · 30 references
Computer Science

TL;DR

It is suggested that early experience leaves a persistent geometric imprint on how neural networks adapt to future tasks.

Abstract

Representations continually change as a network learns new tasks. We ask whether early representational changes naturally form a geometric structure that continues to shape later learning. We identify a low-dimensional subspace of early representation drift, which we call a scaffold, and test whether it is reused across subsequent tasks. Across four pretrained visual encoders and two datasets, later representational changes consistently favor this early-defined subspace over matched random alternatives. This reuse is history-dependent: when networks experience different early tasks but identical later training inputs, each network preferentially reuses the scaffold induced by its own learning history. The same preference appears in individual optimizer updates, even though the network's dominant local response directions shift away from the original scaffold. Finally, constraining motion within the scaffold slows new-task acquisition more than matched random constraints, while effects on old-task retention are less consistent. In summary, these results suggest that early experience leaves a persistent geometric imprint on how neural networks adapt to future tasks.Code is available at https://github.com/YuantaoDeng/latent-scaffold.

View source

Similar papers

Open access Oct 2026

Lateral recurrence dynamically refines visual representations for task performance

Recurrent connections are pervasive in visual cortex, and visual representations evolve as recurrent processing unfolds. Yet the mechanism underlying this temporal transformation remains poorly understood. To address this question, we trained recurrent convolutional networks on face and object recognition and tested th...

Amirhossein Farzmahdi, Hossein Adeli, Bo-Ran Wang et al. · 0 citations

Learn Here, Move Less Elsewhere: Input-Conditioned Plasticity from Retained-Domain Activation Atlases

ATLAS is introduced, which turns retained-domain representations into an input-dependent rule for task adaptation, and provides a practical mechanism for acquiring specialized skills while maintaining continuity in existing responses.

Jiang-Tao Lin, Bang-Yang Wei, Yi-Hang Ding et al. · 0 citations
#machine learning Preprint Nov 2026

Invariance before structure: two modes of emergent compression in representation learning

Learned representations show striking invariance across biological and artificial neural networks, with different substrates and variations driven by drift, heterogeneity and initialisation. To understand this invariance, we propose Emergent Adaptive Representation Learning (EARL), a framework that distinguishes two mo...

Hardik Rajpal, Dan F. M. Goodman · 0 citations

Reliable Replay through Spatial Coherence in Online Continual Learning

This work introduces SPatial coHErent risk control for REplay (SPHERE), a general replay-allocation method applicable across a broad range of learning settings and demonstrates that SPHERE improves accuracy and reduces forgetting across noisy-label vision tasks, continual language-model instruction tuning, and code-gen...

Hai-Xiang Sun, Jie-Fu Zhang, Ying-Hao He et al. · 0 citations

Learning Dynamics of Continual Learning: A Unified View of Data Attribution, Forgetting, and Plasticity Loss

This work derives a token- and layer-wise decomposition of how learning from one token changes another prediction, and connects data attribution, forgetting, and plasticity loss as distinct regimes of the same evolving learning dynamics.

Yi Ren, Wen-Long Deng, Guan-Zhe Hong et al. · 1 citation
#machine learning Preprint Oct 2026

Neural Fields Encode Adaptation Geometry

Neural fields are usually evaluated by how well they reconstruct an observation. We show that this misses two useful properties of a fitted network: how easily it can adapt to new observations, and what its weights retain from earlier ones. We study these properties as adaptation geometry. For images, we meta-learn cla...

Prateik Sinha, Stefania Druga · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.