Large language model (LLM) routing distributes inference work across different models, advancing the cost-quality Pareto frontier of LLM serving. While coarse-grained routing at the session or query level has been widely adopted in production systems, recent algorithmic work shows that fine-grained token-level routing...
Tianyu Fu, Teng-Xuan Liu, Ruo-Xin Wang et al.· 0 citations
Looped transformers have demonstrated promising parameter efficiency by reusing layers for latent computation. Prior studies compare looped and non-looped models at matched parameters or per-token FLOPs. However, to the best of our knowledge, whether looping improves test-time scaling as outputs grow longer remains und...
Yi-Chen You, Tianyu Fu, Ao-Song Feng et al.· 0 citations
This work proposes Think-at-Hard (TaH), a looped transformer optimized for selective iteration that employs a lightweight neural decider to trigger latent iteration, only at tokens likely to be incorrect after the standard forward pass.
Tianyu Fu, Yichen You, Ze-Kai Chen et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.