Skip to content
Conference Open access

Reinforcement Learning for the Working Memory Controller

Sep 2026 · 2026 7th International Conference on Smart Electronics and Communication (ICOSEC) · pp. 394-400 · 0 citations · 18 references

Abstract

Agentic systems built on large language models rely on a bounded-capacity working memory (WM) that holds the small set of items recent observations, active facts, in-progress subgoals immediately available for reasoning, while less pressing information is evicted or consolidated to a long-term store. Existing WM controllers rely on fixed heuristics such as recency (LRU), first-in-first-out ordering, or hand-tuned consolidation scores, all of which optimize a proxy signal rather than the downstream task objective the memory ultimately serves. This paper proposes WMC-RL, a working-memory controller trained with reinforcement learning that formulates admission, eviction, and consolidation as actions in a Markov decision process and learns a policy, via Proximal Policy Optimization, that directly optimizes a composite reward combining sparse task success with dense penalties for retrieval cost, memory churn, and unnecessary consolidation. WMC-RL is evaluated on LongHorizon-30, a 30-task benchmark of long-horizon agent episodes spanning multi-session research assistance, iterative code debugging, and long-document question answering, against random, FIFO, LRU, and a hand-tuned heuristic consolidation-score baseline. WMC-RL achieves a task success rate of 86.1%, exceeding the strongest heuristic baseline by 11.8 percentage points, reduces eviction regret by 53.2%, and improves consolidation precision to 79.8%. Complexity analysis, an ablation study, a sensitivity analysis over the reward-weighting coefficient, and a working-memory-capacity scalability study are reported, together with a qualitative case study of a learned consolidation decision and an explicit discussion of threats to validity.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.