$\sigma$Transfer: Uncertainty Transfer from Small to Large Networks under $\mu\mathrm{P}$
Reliable predictive uncertainty in Laplace approximations depends critically on the prior precision, yet selecting it requires a posterior sweep that is prohibitively expensive for neural networks with billions of parameters. Under the Maximal Update Parametrization ($\mu\mathrm{P}$), we derive a rescaling of the prior...
Richard Bergna, Fernando Ruiz Mazo, Nicolò Felicioni et al.
· 0 citations