A weight space network (or metanetwork) takes the weights of another neural network as input and predicts properties of it. Most prior work trains such models on input networks of one or a few fixed sizes and evaluates them in-distribution. The few attempts at out-of-distribution size generalization remain limited in s...
Yu-Xin Ma, A. Dayan, Yam Eitan et al.· 0 citations
Weight-space networks operate directly on parameters of other neural networks, enabling tasks such as predicting model properties, editing trained models, and generating weights. Weight-space symmetries such as neuron permutations make equivariance a key design principle. However, existing equivariant weight-space arch...
This work studies when finite probe-based representations are sufficient for learning neural functionals, establishes general identification and universality results for probing, and shows that using intermediate hidden representations can provide significantly more informative representations than relying only on fina...
Soutrik Sarangi, Yonatan Sverdlov, A. Dayan et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.