This work introduces RECON (Reasoning over Extended Contexts with Obfuscated Narratives), a benchmark for evaluating compositional reasoning over long contexts and reveals substantial limitations across current architectures.
Abstract
Large language models and LLM-based agents are widely used as personal chat assistants, enterprise copilots, and autonomous workflow agents. In all these applications, memory (the ability to retain, access, and reason over information accumulated over long contexts and multiple interactions) plays a crucial role in determining the reliability of any agent. We introduce RECON (Reasoning over Extended Contexts with Obfuscated Narratives), a benchmark for evaluating compositional reasoning over long contexts. RECON spans 24 case files across three domains (criminal, medical, and financial), each ranging from 50k to 100k tokens, and tests agents on six memory intensive tasks: reconstructing multi-hop evidence chains, propagating cascading invalidations, resolving source conflicts, counterfactual reasoning, satisfying temporal constraints, and temporal fact retrieval. Recent memory benchmarks evaluate whether agents can retrieve scattered facts or detect if a fact has changed whereas RECON evaluates what happens after the change, whether agents can trace which downstream conclusions are affected, which survive through independent support, and how alternative timelines would have unfolded. Our evaluation reveals substantial limitations across current architectures: even the strongest non-Oracle system reaches only 22.4% Accuracy, with retrieval and reasoning each surfacing as challenges.
SodaMem is presented, an evidence-grounded temporal graph memory that extracts typed FactEvents with mandatory provenance spans, persists mention time, occurrence time, and validity with SUPERSEDES/CONTRADICTS/UPDATES edges under hybrid lexical-dense indexing and answers via a planner-reader loop that gathers citable evidence before composing a final response.
This work introduces SLEUTH, which makes an agent's investigative state explicit and actionable through a structured epistemic working memory, and shows that how an agent organizes its reasoning, not raw model capability, is the active ingredient for scaling multi-hop reasoning.
Large language model (LLM)-based search agents answer questions through multi-step interactions with external environments. However, providing complete execution trajectories to the LLM causes unbounded context growth and introduces noise. Existing compression methods reduce context at the cost of important details and often replace erroneous facts without repairing downstream reasoning derived from them. To address this problem, we propose ReTree, a self-correcting tree-structured memory mechanism for search agents. ReTree constructs a bounded per-step reasoning context while preserving source-linked evidence. It models search as an evidence tree whose nodes store bounded summaries, evidence, and revision histories. When newly retrieved evidence contradicts an earlier claim, ReTree traces back to the node where the claim was introduced, replaces outdated evidence, regenerates summaries, prunes affected branches, and resumes search. Source-grounded evidence provenance supports reliable conflict localization and keeps final claims traceable to retrieved passages. Experiments on four public question-answering and search benchmarks show that ReTree consistently outperforms Full-Trajectory ReAct, improving answer accuracy by up to 25.6 percentage points (pp); the average maximum per-step reasoning context of Full-Trajectory ReAct is $1.27$--$1.51\times$ that of ReTree. These results establish ReTree as an effective self-correcting memory abstraction for long-horizon search.
Aijun Yang, Qianxue Guo, Ziyi Huang et al.· 0 citations
ChronoMem is the first open-source system and benchmark for systematic semantic global memory rollback in LLM agents, and a post-exposure evaluation protocol that tests whether an agent can behave counterfactually after rollback by answering queries and summarizing history as if future updates had never occurred.
Yongye Su, Wujiang Xu, Chaoji Zuo et al.· 1 citation
The results suggest that effective long-horizon agent memory depends less on storing more information than on deciding which information should remain active, and that effective long-horizon agent memory depends less on storing more information than on deciding which information should remain active.
This framework proposes a context-editing framework that performs selective abstraction over entities that appear in both the context and the question, establishing symbolic abstraction as a highly cost-efficient solution for ensuring context fidelity in LLMs.
Rounak Sharma, Debabrata Mahapatra, S. Saini· Annual International ACM SIG...· 0 citations