This work introduces a compositional sparse embedding (CoSE) for task conditioning that reduces learned task-memory parameters by over 90% while improving pass@2 in controlled ARC ablations.
Abstract
Recurrent models trained from scratch have recently become competitive on ARC-style reasoning tasks, but the usual framing around small recurrent backbones overlooks two important parts of the system: task-conditioned memory and synthetic augmentation data. We study this regime through CHARM, a compact hybrid ARC model that combines recurrent reasoning with structured task memory, synthetic data, and inference-time aggregation. In existing approaches, task-conditioned memory supplies a large hidden source of capacity, reaching more than 30x the size of the recurrent backbone. We introduce a compositional sparse embedding (CoSE) for task conditioning that reduces learned task-memory parameters by over 90% while improving pass@2 in controlled ARC ablations. For the recurrent backbone, recurrent depth helps only when balanced with learning horizon. Combining these ingredients, our system reaches 84% pass@2 on ARC-AGI-1 and 46.7% pass@2 on ARC-AGI-2 public evaluation. The benefits of structured memory also generalize to unseen puzzles and other domains. Our code, dataset, and model checkpoints are available at https://github.com/water-vapor/charm.
DLMR is a parameter-efficient mechanism that equips MLLMs with Dual Latent Memories: a visual memory that compresses image evidence and a reasoning memory that tracks intermediate conclusions and constraints with only a small number of additional trainable parameters.
Hao-Xuan Ma, Jin-Fei Qi, Yi-Cheng Xiao et al.· 0 citations
This work introduces WISE (Working-set Inference with Support Exploitation), a training-free method that uses unrestricted attention during early recurrent steps to discover a block-structured working set, then reuses its support in later steps while keeping attention weights and recurrent refinement dynamic.
This work revisits Unitary recurrent networks (uRNNs) and proposes three different versions of uRNNs as drop-in replacements for recurrent PPO architectures, and demonstrates that the simple recurrence and the added degree of freedom from the phase of the complex representations enable significant gains over baselines...
Sathya Kamesh Bhethanabhotla, E. Gavves, André Biedenkapp· arXiv.org· 0 citations
Attention with Routed Memory is proposed a novel KV caching structure that introduces a fully differentiable, fixed-size memory system organized as a hierarchical router that achieves superior performance and efficiency compared to fixed KV-caching approaches, while remaining efficient and scalable in terms of both mem...
Qiu-Hao Zeng, Jerry M. Huang, Peng Lu et al.· arXiv.org· 0 citations
RPMem is introduced, a two-stage architecture that compiles each session into a model-independent latent memory through forward computation and selectively integrates it with retained memory via a task-trained recurrent gate, establishing RPMem as a lifecycle-independent parametric memory framework that maintains evolv...
Long-horizon agents compact their history to continue within a finite context window, but a textual summary alone may not support every subsequent decision. We introduce REMORY, a neural memory network that supplements the summary with a bounded sequence of soft memory tokens. Given the history and summary, the network...
Han-Chen Xia, Bao-You Chen, Yu-Tang Ge et al.· 0 citations