Skip to content

Coding Agent Memory Post-training: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via Reinforcement Learning

Sep 2026 · arXiv.org · Vol abs/2609.34422 · 1 citation · 87 references
Computer Science

TL;DR

Coding Agent Memory Gym (CAMG), a suite of long-horizon agentic-RL environments spanning Shop, Coding, DeepResearch, and AutoResearch, which provides executable shell access and an episode-persistent workspace, enabling agents to create, revise, search, and reuse files as memory throughout an episode.

Abstract

Language-model agents increasingly tackle long-horizon tasks whose interaction histories exceed the model's active context. Recent work has begun to use reinforcement learning to make memory control part of the policy, often relying on predefined memory tools within domain-specific training environments of relatively short horizons. This setup ties learned memory behavior to environment-specific interfaces that lie outside the base model's pre-training and must be learned from scratch, so even after post-training, agents struggle to use memory in long-horizon tasks. To address these limitations, we introduce Coding Agent Memory Gym (CAMG), a suite of long-horizon agentic-RL environments spanning Shop, Coding, DeepResearch, and AutoResearch. Alongside each environment's native task interface, CAMG provides executable shell access and an episode-persistent workspace, enabling agents to create, revise, search, and reuse files as memory throughout an episode. We also introduce CAMG-RL, which trains a single policy jointly across all four environments with fully asynchronous PPO, learning this file-based memory behavior directly from downstream task reward, and we train CAMG-RL-4B and CAMG-RL-9B from Qwen3.5 models of matching size. On SWE-bench Verified and MLE-bench Lite, CAMG-RL-4B and CAMG-RL-9B are competitive with Qwen3.5-35B-A3B and Qwen3.5-122B-A10B, respectively.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents

All-in-One Memory (AiO-Mem), which organizes these operations as transitions over a shared evidence state and directly evaluates the semantic quality of each action in a unified reward space, consistently improves answer accuracy over both static memory systems and RL-trained memory agents.

Bing-Bing Wang, Jing-Jie Lin, Wei-Ming Qiao et al. · 0 citations
#artificial intelligence Preprint Oct 2026

METTA: Memory-Enhanced Test-Time Adaptation for Meta-Reinforcement Learning

Meta-reinforcement learning aims to train agents that adapt quickly to unseen tasks from limited interaction, but this requires representing task evidence over long interaction histories. Classical recurrent agents compress experience into hidden states, while transformer-based agents retrieve from past trajectories at...

Ali Najar, Hosein Hasani, Amir Kooshan Fattah et al. · 0 citations
#artificial intelligence Preprint Oct 2026

AMBER: Training Long-Horizon Web Agents through Append-Only Memory

Modern language-model agents increasingly interact with external environments over long-horizon, multi-step trajectories, where the accumulated interaction history can quickly exceed practical context budgets. To ensure reliability, agents must maintain factual information over long horizons, remember execution errors...

Chinmay Savadikar, Zhao-Yu Zhang, Ming-Yu Zhao et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Interactive Memory Learning for Long-Term Conversations

InteraCtive Memory Learning is proposed, a multi-agent framework that transforms the memory mechanism from a passive archive into a learnable, interactive memory policy that significantly outperforms strong baselines.

Cai Ke, Jiang-Yue Yan, Han Zhang et al. · 1 citation
Preprint Oct 2026

Uni-Mem: Learning a Long-Horizon Agentic Memory Manager with Unified Online Policy Optimization

Long-term agent memory is shifting from predefined workflows toward learnable policies for memory construction and retrieval. Existing learning-based methods either optimize construction alone or combine retrieval-side online reinforcement learning (RL) with offline construction supervision, limiting co-adaptation betw...

Zhi-Qian Xia, Jie-Long Tang, Ling-An Zeng et al. · 0 citations
Conference Open access Sep 2026

Reinforcement Learning for the Working Memory Controller

Agentic systems built on large language models rely on a bounded-capacity working memory (WM) that holds the small set of items recent observations, active facts, in-progress subgoals immediately available for reasoning, while less pressing information is evicted or consolidated to a long-term store. Existing WM contro...

J. Gokulraj, L. Karthikeyan · 0 citations

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.