A novel cognitively-inspired hybrid model is presented which optimally trades off simulation and reasoning to improve model performance and computational efficiency and draws a parallel with this to resource-rational theories of human cognition, which propose that people optimally trade off simulation and reasoning dep...
Ying-Qiao Wang, Eric J. Bigelow, Tomer D. Ullman et al.· 0 citations
Recurrent neural networks typically compress information into a single vector-valued recurrent state, forcing short-term computation and long-term retention to share the same representation. Past extensions alleviate this bottleneck by increasing the memory capacity or separating timescales, but lack the combination of...
Ciaran Regan, K. Arulkumaran, L. Darlow et al.· 0 citations
Across Sokoban and autoregressive FineWeb language modeling, this work finds that temporal recurrence shifts the best observed compute allocation toward substantially fewer layers, with comparable or better performance.
Ivan Anokhin, Johan S. Obando-Ceron, Irina Rish et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.