Neural networks whose inputs are functions, as in operator learning, are usually shown to be universal by existence arguments, either by reduction to finite-dimensional networks or by Stone--Weierstrass-type density; these prove that approximants exist but say little about how the parameters depend on the target. We ta...
Sho Sonoda, Luca Galimberti, Yuan-Chao Xu et al.· 0 citations
The message is that the statistical cost of depth is the metric entropy of the set $B(k,F)$ of compositions, and it separates the models whose estimation error is independent of the depth from those whose estimation error grows with it.
Sho Sonoda, Yuka Hashimoto, Isao Ishikawa et al.· 0 citations
This work model SGD gradient noise by adopting a Young function that preserves the finiteness of all polynomial moments while allowing tails heavier than sub-Weibull, including lognormal distributions, and establishes concentration inequalities for $\beta$-heavy-tailed noise.