Fernando Ruiz Mazo

$\sigma$Transfer: Uncertainty Transfer from Small to Large Networks under $\mu\mathrm{P}$

Reliable predictive uncertainty in Laplace approximations depends critically on the prior precision, yet selecting it requires a posterior sweep that is prohibitively expensive for neural networks with billions of parameters. Under the Maximal Update Parametrization ($\mu\mathrm{P}$), we derive a rescaling of the prior...