Skip to content

Author

Neehal Tumma

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Preprint Oct 2026

ParaTTT: Parallelizing Test-Time Training Layers with Deep Memories

Test-time training (TTT) layers replace the linear fast-weight memory of modern recurrent architectures with a small neural network trained online by gradient descent. This nonlinear memory is more expressive than the linear memories of Gated DeltaNet and Mamba-2, but is difficult to parallelize: existing implementatio...

Noel Loo, Neehal Tumma, T. Konstantin Rusch et al. · 1 citation
Preprint Nov 2026

Quantizing Recurrent Language Models Is Easier Than You Think

Linear recurrences are becoming an increasingly common building block in large language models, replacing the growing KV cache of softmax attention with a fixed-size recurrent state that enables efficient decoding. However, the quantization stack has largely been developed around softmax Transformers, whose numerical p...

Philipp Nazari, Neehal Tumma, D. Rus et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.