Skip to content

T-LoopFormer: Token-Level Elastic-Depth Looped Transformers for Latent Reasoning with Dynamic Routing

Sep 2026 · 2 citations · 49 references
Computer Science

TL;DR

This work proposes dynamic token-choice routing for looped transformers, enabling each token to adaptively determine its own number of loop iterations based on its hidden state, which can improve the token generation accuracy and validate the effectiveness of token-choice router and recursion-wise KV cache.

Abstract

Looped Transformers have recently demonstrated strong performance in both reasoning and language tasks by reusing a shared set of parameters across multiple iterations, achieving parameter efficiency without sacrificing representational power. Besides, looped Transformers perform inference directly in the latent space (latent reasoning) to reduce the number of tokens consumed during inference, thereby achieving improved sample efficiency. However, these models typically apply a fixed recursion depth uniformly to every token, leading to suboptimal compute allocation and leaving significant efficiency gains on the table. In this work, we propose dynamic token-choice routing for looped transformers, enabling each token to adaptively determine its own number of loop iterations based on its hidden state, which can improve the token generation accuracy. Moreover, we further introduce recursion-wise KV cache, which maintains an independent key-value cache for each recursion loop, this design ensures that tokens at different depths only attend to their corresponding cached states, effectively enabling faster autoregressive decoding. Extensive experiments show that T-LoopFormer achieves robust performance on language modeling and zero-shot reasoning tasks and our model can reach the lowest decoding latency, which validate the effectiveness of token-choice router and recursion-wise KV cache. Code is available at https://github.com/YuMingQian1234/T-LoopFormer

View source

Similar papers

Improving Test-Time Scaling with Adaptive Looped Transformers

Looped transformers have demonstrated promising parameter efficiency by reusing layers for latent computation. Prior studies compare looped and non-looped models at matched parameters or per-token FLOPs. However, to the best of our knowledge, whether looping improves test-time scaling as outputs grow longer remains und...

Yi-Chen You, Tianyu Fu, Ao-Song Feng et al. · 4 citations

LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers

This work proposes LoopSpec, a training-free self-speculative decoding framework tailored for Looped Transformers that introduces a selective second proposal from deeper recurrent depth while ensuring lossless decoding under both greedy and sampling regimes.

SangLyul Cho, Lang-Qing Cui, Sehoon Kim et al. · 7 citations · ⚡1
Preprint Aug 2026

LoopMTP: A looped transformer guided by latent multi-token prediction

Multi-token prediction (MTP) supplies exactly the dense, forward-looking supervision the loop is missing, by softly aligning the hidden state of loop $t$ with the embedding of the token $t$ steps ahead, while a lightweight gate preserves useful information across iterations.

Behzad Shomali, Markus Frey, D. Berghaus et al. · 3 citations

Looped Transformers as Optimizers

This framework derives loop transitions in closed form from a projection, a local objective and an optimizer update rule, which combines explicit weight decay, adaptive step size and a delta objective and improves average generative performance over the compared looped and non-looped baselines under matched training FL...

Yu-Long Huang, Chen-Yu Jiang, Zhan-Peng Zhou et al. · 0 citations
#machine learning Preprint Sep 2026

FlashLoop: Fast and Memory-Efficient Looped Transformers via Lazy Updates

FlashLoop is introduced, a training-free inference framework that reduces cross-loop redundancy through token-sparse updates, sparse attention, and KV-residual quantization, substantially improving the practicality of scaling Looped Transformers to greater computational depths and longer context.

Wan-Qi Yang, Shi-Wei Liu · 0 citations
#machine learning Preprint Sep 2026

Trading Depth for Time in Recurrent Transformers

The results suggest that temporal thinking offers a parameter-efficient alternative to increasing physical depth in recurrent Transformers, and that temporal thinking offers a parameter-efficient alternative to increasing physical depth in recurrent Transformers.

Ze-Yi Huang, Xuehai He, Yong Jae Lee et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.