It is suggested that early experience leaves a persistent geometric imprint on how neural networks adapt to future tasks.
Abstract
Representations continually change as a network learns new tasks. We ask whether early representational changes naturally form a geometric structure that continues to shape later learning. We identify a low-dimensional subspace of early representation drift, which we call a scaffold, and test whether it is reused across subsequent tasks. Across four pretrained visual encoders and two datasets, later representational changes consistently favor this early-defined subspace over matched random alternatives. This reuse is history-dependent: when networks experience different early tasks but identical later training inputs, each network preferentially reuses the scaffold induced by its own learning history. The same preference appears in individual optimizer updates, even though the network's dominant local response directions shift away from the original scaffold. Finally, constraining motion within the scaffold slows new-task acquisition more than matched random constraints, while effects on old-task retention are less consistent. In summary, these results suggest that early experience leaves a persistent geometric imprint on how neural networks adapt to future tasks.Code is available at https://github.com/YuantaoDeng/latent-scaffold.
Recurrent connections are pervasive in visual cortex, and visual representations evolve as recurrent processing unfolds. Yet the mechanism underlying this temporal transformation remains poorly understood. To address this question, we trained recurrent convolutional networks on face and object recognition and tested th...
Amirhossein Farzmahdi, Hossein Adeli, Bo-Ran Wang et al.· bioRxiv· 0 citations
ATLAS is introduced, which turns retained-domain representations into an input-dependent rule for task adaptation, and provides a practical mechanism for acquiring specialized skills while maintaining continuity in existing responses.
Learned representations show striking invariance across biological and artificial neural networks, with different substrates and variations driven by drift, heterogeneity and initialisation. To understand this invariance, we propose Emergent Adaptive Representation Learning (EARL), a framework that distinguishes two mo...
This work introduces SPatial coHErent risk control for REplay (SPHERE), a general replay-allocation method applicable across a broad range of learning settings and demonstrates that SPHERE improves accuracy and reduces forgetting across noisy-label vision tasks, continual language-model instruction tuning, and code-gen...
Hai-Xiang Sun, Jie-Fu Zhang, Ying-Hao He et al.· arXiv.org· 0 citations
This work derives a token- and layer-wise decomposition of how learning from one token changes another prediction, and connects data attribution, forgetting, and plasticity loss as distinct regimes of the same evolving learning dynamics.
Yi Ren, Wen-Long Deng, Guan-Zhe Hong et al.· arXiv.org· 1 citation
Neural fields are usually evaluated by how well they reconstruct an observation. We show that this misses two useful properties of a fitted network: how easily it can adapt to new observations, and what its weights retain from earlier ones. We study these properties as adaptation geometry. For images, we meta-learn cla...
Prateik Sinha, Stefania Druga· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026