Skip to content
#natural language processing Conference Open access

Grounding Memory Summarization in Utility Intent

Sep 2026 · Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing · Vol abs/2609.33417 · 0 citations · 37 references
Computer Science

TL;DR

This work proposes MemSuit, a self-distillation framework in which a teacher summarizer, conditioned on observed query-answer pairs, produces utility-aware memory entries that a student learns to reproduce from the raw conversation alone, confirming the value of grounding memory in downstream utility.

Abstract

Existing summarizers for memory systems are typically optimized for human-facing criteria such as faithfulness, which misaligns with their true objective: preserving the evidence needed to support future queries. We show that conditioning summarization on query-answer pairs substantially improves answer quality, and that this utility-aware behavior is transferable across queries. Motivated by these findings, we propose MemSuit, a self-distillation framework in which a teacher summarizer, conditioned on observed query-answer pairs, produces utility-aware memory entries that a student learns to reproduce from the raw conversation alone. To prevent collateral erasure where conditioning on a single query-answer pair discards evidence relevant to other plausible queries, the teacher decomposes each block into multiple self-contained entries that preserve distinct query-relevant facets as independently retrievable units. To align the retriever with the compact, fact-dense style of teacher entries, we further fine-tune the embedding model with a contrastive objective supervised by teacher entries. Across a diverse suite of conversational query types, MemSuit consistently outperforms state-of-the-art baselines, confirming the value of grounding memory in downstream utility.

Read PDF

Similar papers

#artificial intelligence Open access Sep 2026

CompCQR: Compositional Query Generation for Training-Free Conversational Search

This work proposes a novel training-free method that generates a very large number of queries with minimal LLM usage by compositionally combining a small set of atomic components and applies LLM reasoning to construct a high-quality document set that balances precision and recall while capturing the user's core intent.

Yunah Jang, Kang-il Lee, Joongbo Shin et al. · 0 citations
#artificial intelligence Preprint Sep 2026

DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents

This work introduces DocHop, a benchmark for integrated chart--context reasoning in document-style images and constructs DocHop via a stochastic logic-first generation pipeline with controllable reasoning depth and visual density, to enable systematic evaluation.

Zhuo-Ran Yu, Le Thien Phuc Nguyen, Jaden Park et al. · 1 citation

NoteVQA: Benchmarking VLMs on Real-Life Questions from Human Communities

This work introduces AgenticInterleave, a single-agent ReAct framework for retrieval-supported answer generation, together with IVR-12, a 12-dimensional rubric for assessing the content, presentation, and image quality of interleaved references and model outputs, and evaluates both short-answer correctness and interlea...

Hao-Nan Jiang, Guo-Jian Zhan, Jian-Cong Xie et al. · 0 citations
#artificial intelligence Preprint Sep 2026

When Users Don't Ask: Benchmarking Context-Driven Memory Retrieval in Conversational Agents

Large language models (LLMs) are increas- ingly deployed as long-horizon conversational agents, motivating growing interest in mem- ory systems. However, existing benchmarks primarily evaluate memory through QA-style probing rather than in-situ conversational usage. We introduce LOCOMO-CONV, a conversa- tional memory b...

Wen-Yu Chang, Yun-Nung Chen · 3 citations · ⚡1

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.