Neural Approximation by Function Composition: Rigidity and Doubly Exponential Convergence
A rigidity theorem is established: for continuous piecewise linear generators with a finite number of segments, any \(C^3\) function that can be represented in this way is at most quadratic, which demonstrates how generator dynamics and remainder estimates govern depth allocation and approximation rates of deep neural...