Representation Complexity Along Learning Trajectories: Trading Width for Precision and Time
What computational role does overparameterisation play during learning? Information-plane analyses suggest that neural networks can traverse representations substantially more complex than those required by their final predictors. We explore when this transient complexity serves a computational purpose by comparing the...
Ryan Singh, Miguel De Llanza, Christopher L. Buckley
· 0 citations