Skip to content

HasMem: Hard-Origin Adaptively Softened Memory for Long-Term LLM Agents

Sep 2026 · 0 citations · 54 references
Computer Science

TL;DR

This work proposes Hard-Origin Adaptively Softened Memory (HasMem), a framework for Frozen hard-prompt embeddings that provides a verifiable initial state for reuse of past interactions in text-based memory and context compression.

Abstract

Text-based memory and context compression support reuse of past interactions. Resizing continuous memory changes the input to a frozen LLM, coupling capacity allocation with readout. We propose Hard-Origin Adaptively Softened Memory (HasMem). Frozen hard-prompt embeddings provide a verifiable initial state. A controller adjusts memory widths, a Writer re-encodes resized entries, and Reader and Global provide readout adaptation and cross-turn state. On all $535$ questions in a reconstruction probe derived from the Multi-Session Chat (MSC) development split, the main configuration achieves lexical F1 of $95.3$ ($+4.4$ percentage points) at $93.6\%$ of the hard reference's framed memory positions. With approximately matched per-question target body budgets, six configurations at mean per-entry retention around $0.83$--$0.91$ exceed rule-based re-encoding by $8.0$--$23.6$ exact-match (EM) percentage points. With fixed model parameters and rule target width ratio $0.75$, Global's EM gain passes a user-level exact paired test with Bonferroni correction over eight comparisons. On all $500$ LongMemEval-S questions, local lexical F1 rises from the hard reference's $3.4$ to $8.9$, and answer negative log-likelihood (NLL) falls from $12.257$ to $5.274$. F1 gains accompany lower EM on both evaluations.

View source

Similar papers

#natural language process... Preprint Oct 2026

Writability and the Pathway: A Controlled Study of Knowledge Injection into Addressed Memory

Addressed conditional memory provides a local surface for post-hoc knowledge injection, but a successful write need not transfer to a new question or preserve untargeted behavior. This study combines a controlled GPT-2 comparison with production-memory experiments on Qwen3.8-Flash-Next-FP8 and DeepSeek-V4.1-Flash. Newl...

Wei Gao, Cheng Tian, Ya-Wen Duan et al. · 1 citation

The Evolution of Attention in Large Language Models: Mechanisms, Trade-offs, and Emerging Trends

Self-attention gives LLMs fine-grained, query-dependent access to context, but dense token interactions incur quadratic prefill cost and a key--value cache growing with context length. Research thus spans explicit-memory compression, sparse access, recurrent state construction, structured state dynamics, and heterogene...

Zhen-Tao Tan, Jing-Yi Shen, Yan-Bo Li et al. · 0 citations

ARM: Attention with Routed-Memory for Learnable Sparse Control

Attention with Routed Memory is proposed a novel KV caching structure that introduces a fully differentiable, fixed-size memory system organized as a hierarchical router that achieves superior performance and efficiency compared to fixed KV-caching approaches, while remaining efficient and scalable in terms of both mem...

Qiu-Hao Zeng, Jerry M. Huang, Peng Lu et al. · 0 citations
#natural language process... Preprint Oct 2026

Hy-Memory-30B-A3B: Attributed Reinforcement Learning for Long-Term Memory Writing

External-memory systems extend language-model applications beyond a context window, but their quality depends critically on the model executing the write path: deciding what to retain, how precisely to represent it, and how new evidence should update persistent state. Existing frameworks typically place a general-purpo...

Tian-Xiang Fei, Jia-Zheng Kang, Mao Zheng et al. · 1 citation · ⚡1
#machine learning Preprint Oct 2026

FOLD: Position-Free Far Memory with Local Order for Long-Context Speculative Decoding

This work proposes FOLD, a training-free method that preserves exact local positions and treats all distant entries as far memory at a shared position and sharpen attention within it to emphasize relevant entries while separately controlling its total weight relative to the local window.

Shu-Xuan Li, Jia-Liang Kang, Yang-Hao Zhou et al. · 0 citations

RPMem: Learning Long-Term Recurrent Parametric Memory Across Sessions for LLM Agents

RPMem is introduced, a two-stage architecture that compiles each session into a model-independent latent memory through forward computation and selectively integrates it with retained memory via a task-trained recurrent gate, establishing RPMem as a lifecycle-independent parametric memory framework that maintains evolv...

Fan-Yu Zhao, Rui-Ke Cao, Liang Dong et al. · 1 citation · ⚡1

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.