Parameter-Efficient Neural Differential Equation Solvers via the Horner Scheme

Abstract

We propose a parameter-efficient neural architecture for solving differential equations that restricts the hypothesis class to Horner-factorized polynomials, yielding an implicit, differentiable solution representation with only a small set of learnable coefficients. For one-dimensional differential equations, initial conditions are enforced exactly by construction through analytical reconstruction of the low-order coefficients, so training reduces to minimizing the differential-equation residual at collocation points. On three benchmark ordinary differential equations, a Horner model with only 9 learnable parameters uses 88.8–91.5% fewer parameters than the considered sigmoid multilayer perceptron, sinusoidal representation network, residual network, and Kolmogorov–Arnold network baselines, and achieves the lowest root-mean-square error in all nine solution- and derivative-level comparisons. Under the adopted training protocols, it also reduces training time by factors ranging from 4.6 to 6.7 relative to the fastest competing baseline, depending on the problem. To improve approximation without abandoning the low-parameter regime, we introduce a piecewise (“spline-like”) extension in which multiple low-order Horner models are defined on subintervals with interface continuity imposed directly by the architecture. The compact model remains accurate under sparse collocation sampling and perturbations of the prescribed initial condition. On a stiff benchmark, a 31-parameter spline-like model reduces the solution and derivative errors by factors of approximately 5.0 and 4.5, respectively, relative to a 100-parameter global Horner model. The construction also extends to multidimensional problems, as demonstrated on a heat-equation example with 45 learnable parameters.