Rodrigo Treviño

Failure of entropic selection for diagonal matrix completion in deep linear networks

We study the free-energy gradient flow associated with the depth-$N$ deep linear network on the space of invertible real $d\times d$ matrices. The loss observes only the diagonal, \[ E_d(W)=\frac12\sum_{i=1}^d(W_{ii}-1)^2, \] and the regularizer is the Boltzmann entropy of the balanced factorization fiber computed by M...