There is a number $\psi$ such that for all $\varepsilon>0$ the probability that the value of the output neuron is in $[\psi - \varepsilon, \psi + \varepsilon]$ tends to 1 as $n$ tends to infinity.
Abstract
We consider for an arbitrary fixed $\rho$ and for each positive integer $n$ a multilayer feedforward artificial neural network with $\rho$ layers, $n$ neurons in the first layer (the input layer) and only one neuron, the output neuron, in the last layer. Very roughly formulated, the main result is that if the distribution of weights of connections from a layer to the next are, for all large $n$, approximated well by a fixed continuous (but otherwise arbitrary) curve which does not depend on $n$, and if the values of the $n$ input neurons are independently and identically distributed with a continuous probability density function, then there is a number $\psi$ such that for all $\varepsilon>0$ the probability that the value of the output neuron is in $[\psi - \varepsilon, \psi + \varepsilon]$ tends to 1 as $n$ tends to infinity.
A global universal approximation theorem in the topology of L^p(\mu) is shown, where any function $f:\mathfrak X\to \mathbb R$ in $L^p(\mu)$ can be approximated to any degree of accuracy by suitable infinite dimensional architectures.
The techniques share most of the high-level ideas presented in [Ruess et al., 2026], but there are also some minor differences which may be of interest for future research on this problem.
We investigate the best $L_2$ approximation of mixed Sobolev spaces by shallow neural networks with $n$ neurons and general activation functions. We first establish an activation-independent Fourier-block principle: if an activation has univariate approximation order $\rho$ in the sense of the Fourier-block property, t...
It is proved that dictionaries obtained by independent parameter sampling from any prescribed density bounded away from zero attain the same approximation exponent with high probability, up to logarithmic oversampling.
A spherical-cap construction proves the latter claim without assuming sparsity merely on the sampling support without assuming sparsity merely on the sampling support, and obtains agnostic minimax excess-risk bounds of order up to logarithms.
Xiao-Yu Li, Zhizhou Sha, Jiao-Jiao Jiang et al.· 0 citations
High-probability bounds for mixed input derivatives of wide random neural networks whose activation derivatives satisfy a factorial growth bound are established and the derivative estimates are connected to quasi-Monte Carlo integration and indicate how such regularity can enter the analysis of QMC-based training.
Josef Dick, Michael Feischl, Fabian Zehetgruber· 0 citations
