The results support generated memory as a selective correction to direct retrieval and highlight routing when, which, and how strongly to intervene as the central challenge.
Abstract
Learned-memory methods store information in an explicit table and consume it through a separate reader, allowing addressing, storage, and reading to be modified independently. We study whether useful memory can also be generated rather than only retrieved. MemoryAthena uses three pathways: direct Engram retrieval (E), generation from retrieved Engram cues (GE), and generation from causal backbone states without consulting the memory table (GH). Generated memory is conditionally useful: it can complement E in one context but interfere with it in another. MemoryAthena therefore treats E as an anchor and learns when a generated representation should intervene. With the backbone, memory, generators, and readers frozen, a lightweight causal routing head is trained from counterfactual future-token likelihood advantages of GE and GH relative to E. At inference time, an admitted candidate modifies the E residual through bounded interpolation, while rejection recovers the direct pathway exactly. On question answering, MemoryAthena raises the five-task average from 37.65 to 39.28 over the direct pathway of the same checkpoint, while the six-task general-NLP average increases from 76.73 to 79.13. The complete memory-side system contains approximately 201M parameters, excluding the frozen backbone. Further analyses show complementary strengths among E, GE, and GH across tasks and inputs. These results support generated memory as a selective correction to direct retrieval and highlight routing when, which, and how strongly to intervene as the central challenge.
This work systematically investigates when, where, and to what extent conditional memory should participate in scientific reasoning, and proposes a Knowledge Boundary-Aware Router that uses task-specific input proxies available before generation to determine whether memory is activated, which layer-stage nodes receive...
Zhen Bi, Xue-Shu Chen, Yan Wang et al.· 1 citation
A controlled harness evaluation of memory substrates for memory-augmented agents, covering dense and sparse indices, text records, structural stores, hierarchical stores, refinement-based memories, parametric updates, and activation-compatible context mechanisms, shows that no single substrate consistently dominates.
Wei-Chieh Huang, Wei-Zhi Zhang, Yu-Chen Wu et al.· 2 citations
The latent cause theory of memory modification provides a computational account of how the brain decides whether to update existing memories or form new ones, but leaves unspecified the neural mechanisms implementing this inference. We propose Reuse-and-Append Memory (RAM), a set of mechanistic principles that achieve...
Together, these results reveal how fixed-size recurrent memory supports selective recall despite shared storage, while exposing the interference and capacity limits that distinguish it from token-addressable KV memory.
Kichang Lee, JaeYeon Park, Songkuk Kim et al.· arXiv.org· 0 citations
GenMem is presented, which reformulates memory management as generative symbolic addressing, a multi-level discrete token tuple drawn from a Cartesian-product address space that factorizes a million-scale sparse memory space using fewer than one hundred discrete symbols.
Xin-Ke Jiang, Tao Feng, Wei-Xuan Xu et al.· arXiv.org· 0 citations
Video models increasingly use memory to preserve information over long sequences, with the assumption that gains come from retrieving and using the correct past content. Standard memory ablations test whether memory helps, but not whether the retrieved content is responsible. We test this directly with read-time memory...
Aditi Tiwari, Akshit Bhalla, D. Prasad et al.· 0 citations