Temporal Recurrence Favors Fewer Layers
Across Sokoban and autoregressive FineWeb language modeling, this work finds that temporal recurrence shifts the best observed compute allocation toward substantially fewer layers, with comparable or better performance.
Ivan Anokhin, Johan S. Obando-Ceron, Irina Rish et al.
· 1 citation