Deep Learning of Compositional Targets with Hierarchical Spectral Methods
TL;DR
This work analyzes compositional target functions of three-layer fitting using an explicit three-layer fitting model trained via layer-wise spectral estimators, and establishes Gaussian comparison bounds for projected moments when $0<\varepsilon<1/3$, and motivates separations in sample complexity between two- and three-layer learning strategies.
Abstract
Why depth yields a computational advantage over shallow methods remains a central open question in learning theory. We study this question in a controlled high-dimensional Gaussian setting, focusing on compositional target functions. We analyze their learnability using an explicit three-layer fitting model trained via layer-wise spectral estimators. The target combines polynomial latent features with a scalar readout, and its compositional structure allows learning to proceed in stages. Recovering an intermediate representation reduces the problem to lower-degree spectral estimation tasks, as in multi-index models. Our analysis establishes Gaussian comparison bounds for projected moments when $0<\varepsilon<1/3$, and motivates separations in sample complexity between two- and three-layer learning strategies, conditional on a conjectured spectral recovery step.
