Prospective Multimodal Memory Compilation is proposed, a framework that shifts part of the memory reasoning process from query time to memory consolidation time and improves answer quality and visual evidence recall while reducing query-time token and latency costs.
Abstract
Long-term memory is essential for LVLM agents to maintain consistency and integrate information across extended multimodal interactions. Existing agent memory systems, however, often reduce visual experiences into textual summaries or rely on static retrieve-then-reason pipelines, which are inefficient at query time and brittle when questions require image-text binding, temporal updates, or visual details. We propose Prospective Multimodal Memory Compilation, a framework that shifts part of the memory reasoning process from query time to memory consolidation time. Given accumulated multimodal interactions, a Questioner predicts future question candidates, a Planner compiles question-conditioned multimodal memory programs, and a Doubter verifies whether the planned evidence path can support the predicted answer. The verified question-program pairs form a structured question bank for efficient query-time routing and evidence retrieval. Experiments on multimodal long-term memory benchmarks show that our method improves answer quality and visual evidence recall while reducing query-time token and latency costs. Extensive ablations analyze the effects of self-feedback, dynamic planning, raw-image access, and question bank coverage.
RIME is introduced, a retrieval-induced memory framework that shifts memory construction from monolithic compression toward evidence-centered integration and consistently achieves the best performance across all three quality metrics among the compared methods, while requiring substantially fewer query-time LLM tokens.
Wan-Qi Zhou, Jia-Wei Lu, Yang Wang et al.· arXiv.org· 0 citations
EM^2Mem is proposed, an event-centric multimodal memory framework that binds heterogeneous evidence to event anchors during memory construction, enabling compact evidence readout over grounded multimodal events rather than modality-specific fragments.
Yijun Chen, Yangfan Zheng, Yanyang Li et al.· 1 citation
The results suggest that effective long-horizon agent memory depends less on storing more information than on deciding which information should remain active, and that effective long-horizon agent memory depends less on storing more information than on deciding which information should remain active.
MEMO is proposed, a multimodal evidence memory organization method for LLM agents that presents memory more efficiently with fewer memory tokens, improves downstream task performance, and builds more effective working memory under constrained budgets.
Xian Gao, Jin-Peng Wang, Jiacheng Ruan et al.· 0 citations
UtilMem is introduced, a diagnostic benchmark comprising 1,717 instances across five domains, designed to evaluate four underexplored aspects of memory utilization: reasoning over dense histories, identifying implicitly relevant memories, synthesizing distributed evidence into summaries, analyses, or plans, and resisti...
Long-term memory systems for large language models (LLMs) have gained popularity for extending reasoning capabilities across applications. Current memory systems rely on LLM agents to organize and consolidate memory, resulting in costly, inefficient write operations. To address this limitation, we propose MemFit, a lon...
Mitchell Piehl, Mu-Chao Ye· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.