How expressive is prompting a transformer? Answering this question is important for separating the roles of prompting, architecture, and pretraining in transformer models, and for determining whether task-specific behavior must be stored in model weights or can instead be induced at inference time through the prompt. W...
This work characterize task-space complexity through covering numbers under a prescribed metric, thereby quantifying the low-dimensional cross-task structure without requiring an explicit parametric representation and explicitly construct a Transformer that exploits their low-dimensional structure to perform ICL.
Zhong-Jie Shi, Rong-Jie Lai, Alexander Cloninger et al.· 1 citation
This work introduces a permutation-equivariant neural operator for trajectory planning of spacecraft swarms, and maps distributions of spacecraft initial states, target states, and obstacle initial states to trajectories which avoid collision and conserve fuel.
Sidhdharth D. Sikka, Suyi Gao, Ze-Hui Lu et al.· 0 citations
Stochastic mean-field control (MFC) provides a fundamental framework for coordinating large populations of interacting agents under uncertainty, with a wide range of applications. Existing numerical and deep-learning methods solve one MFC problem instance at a time and must be re-optimized whenever the task changes. In...
The log-ratio variation $\X_\omega$, the mean absolute pairwise difference of the target-to-pushforward log-density ratio under a weighting measure under a weighting measure, is introduced and used to define KLXX, a new loss function and the Fisher--Rao gradient flow of KLXX is derived, where both variations contribute...
Qi-Han Feng, Rong-Jie Lai, Di Qi et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
