Shivaprasad B. Gowda

Representations Have Trajectories: Characterizing Temporal Representation Dynamics in Neural Networks

Recent work has shown that deep neural networks exhibit an implicit low-rank simplicity bias. In this paper, we investigate the questions that naturally arise from this observation. What are the temporal consequences of low-rank simplicity bias during optimization? How does representation geometry evolve throughout opt...