Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning
Neural Low-Degree Filtering (Neural LoFi) is introduced, a stylized limit of gradient-based training in which hierarchical feature learning becomes an explicit iterative spectral procedure and provides a mathematically explicit framework for studying multi-layer feature learning beyond the lazy regime.