Foundation models for dynamical systems can forecast unseen partial differential equations (PDEs), ordinary differential equations (ODEs), and time series during inference. However, it remains an open question whether these foundation models truly learn generalizable in-context forecasting strategies. We show that loop...
Jeffrey Lai, John Quinn, Anthony Bao et al.· 1 citation
The results show that reasoning slowdowns are an inevitable consequence of problem hardness in modern artificial intelligence models, and establish reasoning traces as a rich new class of dynamical system.
Jeffrey Lai, Anthony Bao, John Quinn et al.· 4 citations
This study develops a theoretical framework framing transformers as kernel regressors, motivating a purely intrinsic strategy for ablating heads based on the stable rank of the per-head projection matrices, and uncovers the specific heads responsible for degenerate phenomena widely observed in TSFMs.
Anthony Bao, Venkata Hasith Vattikuti, Jeffrey Lai et al.· arXiv.org· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.