Stable initialization without the CLT
This work derives the uniform-phase initialization, which obviates distributional approximation and fully decouples the layers in networks with sine activations, and is the first work to use the sine function's periodic symmetry.
Simon Kuang, Kyle R. Chickering, Xin-Fan Lin
· 0 citations