A simple shallow neural network for emulating the solution to singularly perturbed problems
TL;DR
A shallow NN is described which exploits available asymptotic expansions for the solution to singularly perturbed second order boundary value problems, with two small parameters, to augment the approximation space with suitable exponential functions, similar to enriched spaces in finite element methods.
Abstract
We consider (feed-forward) Neural Networks (NNs) for the emulation of the solution to singularly perturbed second order boundary value problems, with two small parameters. We describe a shallow NN which exploits available asymptotic expansions for the solution. These additive decompositions into smooth and layer components, allow for derivative estimates which are explicit in the order of differentiation as well as the singular perturbation parameter(s) \cite{melenk, Irene, SX}. Utilizing such decompositions, we propose a simple NN for emulating the solution to such problems using the $\tanh$ activation function together with different training objectives, such as residual or energy minimization. The key idea is to augment the approximation space with suitable exponential functions, similar to enriched spaces in finite element methods, e.g.~\cite{Kellogg}. Numerical examples in one and two dimensions, including a smooth non-tensor-product domain, illustrate the resulting parameter-robust behavior over the tested perturbation ranges.