MEMTIER, a tiered memory architecture and consolidation framework for an open-source agent runtime and three questions: what to store, what to inject, and what to keep are studied and cast agent memory as a pattern recognition problem: recognizing which session patterns carry evidence and which knowledge types to retain.
Abstract
Long-running large language model (LLM) agents accumulate memory across many sessions, yet most agent runtimes store it as flat text. We present MEMTIER, a tiered memory architecture and consolidation framework for an open-source agent runtime and study three questions: what to store, what to inject, and what to keep. First, a tiered episodic–semantic memory reaches an accuracy of 0.382 and an F1 of 0.412 on LongMemEval-S (N = 500) with a 7B model on a 6 GB GPU—a +33-point gain over no retrieval. A controlled single-pipeline ablation shows the gains come from two components—semantic pre-population and two-stage scoping—while the multi-signal retrieval scaffold is, in this setting, an operational pass-through of BM25 top-k; removing it entirely leaves accuracy unchanged. An oracle analysis explains why the system is retrieval-limited: on the diagnostic subset, the correct session is present in the candidate set 98% of the time, so the primary binding constraint is which evidence reaches the reader. A larger generator helps but does not remove the retrieval bottleneck. Second, the bottleneck is injection granularity, not session recall (90.9% session coverage versus 4.5% fact recall@2). Injecting all facts from the top-k retrieved sessions improves multi-session accuracy by +0.120 and knowledge update accuracy by +0.205, and a controlled comparison isolates a structure effect distinct from token quantity. Third, we cast memory consolidation as a Markov decision process, characterize three structural failure modes that render a learned policy uninformative on single-subject benchmarks, and show preliminary evidence on a live agent benchmark that a typed, keyword-based policy recovers near-oracle performance for one memory type (threat pattern memory) without supervision. Type-dependent retention for other memory types is proposed but not yet evaluated. We frame these as related diagnostic studies rather than a single validated pipeline, and, together, they cast agent memory as a pattern recognition problem: recognizing which session patterns carry evidence and which knowledge types to retain.
Enterprise repositories are large, heteroge- neous, and continuously updated, making re- trieval difficult when efficient access, source- faithful evidence, and cross-query adaptation must be supported together. Enterprise mem- ory extends retrieval beyond the model con- text, but existing systems do not jointly address collection-specific hierarchy construction, low- cost routing, detailed evidence loading, and workload-aware memory updates at this scale. We introduce MEMONDEMAND, short for On- Demand Memory, a memory management sys- tem with three coordinated mechanisms: a dy- namic multi-level hierarchy that determines the abstraction structure and depth for each col- lection, dual memory at every hierarchy level that separates distilled routing from detailed evidence, and on-demand memory promotion that updates node priority under a bounded active-state budget. On EnterpriseRAG-Bench, MEMONDEMAND outperforms the strongest published LB#1 result at every evaluated scale from 10M tokens through the complete 618M- token collection, with gains of 12.23% at 10M and 4.66% at 618M. Results on FinanceBench, HotpotQA, and FRAMES further show strong performance across financial, multi-hop, and fact-retrieval settings. Together, these results establish MEMONDEMAND as an accurate, ef- ficient, and scalable memory solution for very large enterprise repositories across data scales, domains, and evidence requirements. Our code is available at https://github.com/ xfab-xinyuansong/MemOnDemand.git.
Xinyuan Song, Bowen Zhu, H. Haque et al.· 0 citations
This work proposes Akashic, a low-overhead memory system built around MemAttention, which organizes context into bounded chunks and models semantic relationships across chunks, preserving cross-chunk evidence without repeatedly rewriting the full history.
Yang Liu, ZhaoKai Luo, Huayi Jin et al.· 0 citations
Deployed LLM agents increasingly keep their long-term memory as a filesystem: a directory tree of markdown files that the agent itself reads, writes, and reorganizes through generic file tools. Yet research has largely passed over this medium: prior systems design bespoke memory representations and study retrieval over them, leaving the default's two working assumptions untested: that an agent can keep a growing store organized as memories accumulate, conflict, and go stale, and that this organization pays. We present the first systematic exploration of filesystem-based memory for LLM agents. We formalize the setting as three roles around one memory filesystem: a management agent integrates and organizes incoming content, a search agent answers queries with cited sources, and an execution agent supplies task trajectories that are distilled into skills, unifying declarative memory and skills in a single store. Across long-conversation benchmarks and embodied tasks, we vary memory shape (agent-organized hierarchy, verbatim dump, chunk retrieval), stream scale, tool harness (sandboxed shell, memory-tool-style functions, varied search tooling), and the strengths of the management and search agents, tracking answer quality, cost, and store health as memory grows. What organization reliably buys is search economy: organized stores roughly halve retrieval cost where material is large. Today's agents, however, fall short of the default's promise: in our growth study, organization erodes for all but the strongest management agent, and no agent we measure converts organization itself into better answers. And the model is not the only lever over a store's shape: changing the tool set alone reshapes the store as strongly as swapping the model. The study turns the filesystem default from an assumption into a design space for agent memory.
Sizhe Zhou, Sheldon Yu, Hui Wei et al.· 0 citations
Retrieval-augmented generation (RAG) imposes a prefill cost proportional to retrieved context length, and -- with Transformer backbones -- a KV-cache that grows with each generated token. State-Space Models (SSMs) avoid the second cost by construction; we eliminate the first, collapsing prefill from $O(L_{context})$ to $O(1)$ per query. We introduce PRECOG (Pre-Computed Context Injection), a retrieval mechanism that exploits a property unique to SSMs: the fixed-size, position-agnostic recurrent hidden state is a complete summary of everything the model has read. PRECOG pre-encodes document corpora offline as SSM hidden states and injects the best-matching state directly at query time, bypassing in-context re-ingestion entirely. The same state-injection mechanism enables SMC (Structured Memory Consolidation): a hierarchical persistent memory with cognitive-domain clustering, an adjustable fidelity-vs-storage dial, and $O(1)$ session initialization, which consolidates short-term episodic states into long-term semantic memory and fuses both with retrieved corpus states at query time. We demonstrate the system on TENNs-LLM, a 1.2B-parameter gated-SSM language model with a 192 KB hidden state. PRECOG matches in-context RAG answer quality, reducing prefill latency from $\sim$27 s to $<$6 ms on edge hardware -- a $\sim$4500$\times$ speedup that crosses the threshold from unusable to interactive. The mechanism is architecturally impossible for Transformer KV-caches, which are position-entangled and grow linearly with context length.
Anusha Madan, Gopal Aras, Pirbadian Kristofor et al.· 0 citations
This work presents InferScale, a GPU-native LLM memory system that replaces repeated prompt prefilling with reusable KV state, and encodes each memory fact together with a small window of preceding conversation context while caching only the target fact's KV.
This paper argues that Muscle Memory - the practice of compiling recurring user intent into purpose-built specialist agents - is a distinct memory paradigm from retrieval, and argues that compilation is a better fit for the workloads where current assistants impose a multi-turn tax on their users.