Youssef Khouidi

Write Where Short Memory Cannot Reach: Distance-Invariant Fact Recall in Fixed-State Language Models

Fixed-state recurrent language models such as Gated DeltaNet and Mamba promise constant memory at any context length, yet in practice they recall a fact a few thousand tokens later and then lose it. We remove both ways a delta-rule memory forgets, decay and erasure by later writes, on archival heads whose decay is pinn...