This work studies attention-indexed models, a broad framework that can represent multi-layer and multi-head attention architectures, and reveals that attention parameterization itself can act as an architectural implicit bias.
Yi-Zhou Xu, M. Sagitova, L. Zdeborová et al.· 0 citations
Experimental results suggest that the parameters of trained autoencoders can be viewed as sample representations, allowing for a high degree of accuracy in distinguishing between models trained on different data subsets, without resorting to complex vector generation algorithms or using the original samples.
M. Nikitina, Anton Bishuk, O. Bakhteev· 0 citations
Causal foundation models are pretrained neural networks that estimate causal quantities, such as the average treatment effect, on entirely new datasets using in-context learning without requiring model updates.
Christopher Stith, Hossein Rahmani, Jesse C. Cresswell· 0 citations
Finite-Library Input-Warped Bayesian Optimization (FLIWBO), which selects warps from a finite library of smooth input maps by any history-dependent rule, and adapts the input geometry to accelerate learning while retaining high-probability convergence guarantees under mild hypotheses, with an explicit library-size cost...
Edvin Ketabati Augustinsson, Robert A. Bridges· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Learned graph simulators provide an efficient alternative to high-fidelity solvers for granular dynamics. However, granular motion depends strongly on inter-granular contact history, which is difficult to preserve when particle contacts form, break, and rearrange. Existing simulators mainly store temporal information i...
Chang-Jian Zhou, Negin Yousefpour, Jie Qi et al.· 0 citations
Reinforcement learning typically optimizes average reward. For generative policies, the average can hide an important distinction: two policies can achieve the same mean reward while having very different chances of producing a rare but high-reward rollout. This matters as sampling increases during training and inferen...
Shrinivas Ramasubramanian, Daman Arora, Fahim Tajwar et al.· 0 citations
Graph neural networks (GNNs) are a class of neural networks suitable for learning on graph-structured data. Their application to spatial data is a natural extension, however its relatively unclear which message-passing operations, architectural configurations, and graph representation is best suited for classifying cha...
The direction of ignorance is causally active: raising or lowering $\lambda$ at the final prediction state steers the prediction toward or away from the unigram prior in KL divergence, with larger models generally exhibiting lower prior reliance in the high-context limit.
To-Ni J. B. Liu, Jiajun Bao, Yizhou Liu et al.· 0 citations
This analysis quantifies the role of the reweighting factor in the score approximation error, providing theoretical guidance for weighting choices used in practice, and develops a Neural Tangent Kernel analysis tailored to diffusion training with stochastic gradients.
Stanislas Strasman, Sobihan Surendran, S. Le Corff· arXiv.org· 0 citations
A variation space for nonlinear operators between Hilbert spaces, defined through vector-valued Borel measures of bounded variation, is formulated and analyzed, which yields algebraic approximation and learning rates in contrast to the complexity barriers for Lipschitz and Fr\'echet differentiable operator classes.
It is shown that discretized F\"ollmer processes give natural hyper-parameter settings of the DDPM sampler while accommodating a broader class of variance schedules than discretized reverse SDEs.
Yuta Koike· Japanese Journal of Statisti...· 0 citations
This work proposes a Gram-based anisotropic objective perturbation, a ``pre-distortion"strategy that counteracts the distortion from the covariate structure to restore isotropy in the estimation process and provides theoretical insights into designing stable and efficient private estimators without relying on data-depe...
A machine-learning approach called CheMeleon provides a practical way to develop new drugs or other molecules with much less detailed, expensive experimentation.
Is your company’s AI sovereign? A practical guide for CIOs on AI vendor lock-in, EU Data Act switching, cloud sovereignty, and testing your exit plan before you need it. The post Sovereignty Is the Option to Leave appeared first on GPT-Lab.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.