Adaptive versus non-adaptive sampling for Gaussian-RBF surrogates: a replicated benchmark across analytical and differential-equation models

Approaches to nonlinear programming problems: Taylor series expansion, RBF surrogate modeling, DOE-based dimensionality reduction, and adaptive domain splitting

Solving nonlinear constrained optimization problems efficiently remains a challenge. This paper describes a four-component method developed to address these difficulties: Design of Experiments (DOE) screening using L9 and L81 orthogonal arrays to eliminate statistically non-significant variables before optimization begins; adaptive domain splitting into 4–8 gradient-guided subregions to handle multimodality; Taylor series expansion (1st–3rd order) and Radial Basis Function (RBF) surrogate modeling for local and global function approximation respectively; and a hybrid Genetic Algorithm–Sequential Quadratic Programming (GA–SQP) solver. The methodology was evaluated on 20 benchmark nonlinear programming functions spanning low- (1–3 variables), medium- (4–6 variables), and high-dimensional (7–12 variables) problems, as well as 15 practical engineering case studies. DOE-based achieved 95 and 99% confidence in variable selection, reducing problem dimensionality by up to 75% in some cases. RBF surrogates achieved convergence on all 20 benchmark functions, reducing computational time by 70–80% compared to Taylor-series approaches. Adaptive domain splitting reduced function evaluations by 30–35% on multimodal problems. Taken as a whole, the results make a clear case that pre-screening variables with DOE, approximating the reduced function with RBF surrogates, and refining solutions with GA–SQP can cut both computational effort and convergence time substantially without sacrificing accuracy.

Collocation point strategies in physics-informed neural networks: A comparative study on the Burgers' equation

Residual-Christoffel Sampling for Random Feature Collocation of Linear PDEs

Random feature collocation fixes a randomly generated trial space and determines its coefficients from a linear least-squares system. Stability then depends on whether the sampled residual equations represent the geometry induced by the differential operator. We construct an operator-aware discretization in which the operator-applied features determine both the collocation measure and a coefficient whitening map. The randomized scheme combines a residual-Christoffel density with inverse-density weights, while a deterministic scalar-row alternative maximizes successive regularized log-determinant increments. Conditional on the realized trial space, the sampled whitened interior Gram is a spectral approximation to the reference Gram on the retained residual space, with sample complexity linear in the retained dimension up to a logarithmic factor. For uniformly analytic residual kernels, the associated operator has stretched-exponentially decaying eigenvalues and ridge effective dimension that is polylogarithmic in the inverse ridge scale. Experiments on scalar and vector equations, varied geometries, and one to three spatial dimensions show that residual-space sampling and whitening produce numerically full-rank transformed systems with substantially smaller condition numbers and iteration counts. The deterministic construction attains the lowest errors at the smallest scalar sample sizes. Residual-space geometry therefore yields a principled design for stable strong-form random feature collocation.

Adaptive Non-Linear Partition of Unity Methods for Scattered Data Interpolation with Discontinuities

Scattered data approximation with discontinuities is challenging due to the Gibbs phenomenon, which significantly reduces accuracy near interfaces. The recently introduced Non-Linear Partition of Unity Method (NL-PUM) addresses this by combining Radial Basis Function (RBF) interpolation with a non-linear Weighted Essentially Non-Oscillatory (WENO) strategy. While effective, NL-PUM's performance relies heavily on two fixed hyperparameters: the RBF shape parameter and the patch radius. This work extends NL-PUM by adapting both hyperparameters locally using Leave-One-Out Cross-Validation (LOOCV) minimized via Global Optimization with Optimistic Improvement (GOOI). Our main innovation is a smoothness indicator linking a discontinuity-aware shrinkage process to LOOCV-based radius selection: patches in smooth regions remain unchanged, while those near discontinuities automatically shrink to avoid crossing the interface. The resulting method, LOOCV-NL-PUM-GOOI, requires no prior knowledge of interface geometry and introduces no extra cost beyond standard adaptive shape parameter selection. Numerical experiments on synthetic test functions with jump discontinuities and a real-data application to Norwegian Fjords elevation data confirm that this approach substantially reduces approximation error near discontinuities while preserving full accuracy in smooth regions.

A Hybrid PSO–Fifth-Order Iterative Technique for Nonlinear Systems with Applications in Biological Models

Nonlinear systems of equations arise across engineering, physics, and biological modeling; however, classical Newton-type methods may fail when the initial approximation lies outside the convergence region of the NJN local solver. This work proposes a two-stage hybrid framework that couples Particle Swarm Optimization (PSO) for global exploration with the fifth-order Newton–Jarratt (NJN) iterative method for local refinement. The fifth-order convergence of the NJN phase, established through a complete Fréchet-derivative Taylor expansion with explicitly computed error constants, guarantees rapid local convergence once PSO delivers a sufficiently close starting point. The framework is validated on four test problems with increasing numbers of dimensions (n=2,5,20,40): a two-dimensional benchmark algebraic system, a five-dimensional metabolic network model for ethanol production in Saccharomyces cerevisiae, and two large-scale Hammerstein integral equation systems. Over 30 independent runs per method and under the tested conditions, PSO-NJN achieves 100% convergence with mean final residuals of order 10−14–10−16, while pure PSO fails completely on the high-dimensional Hammerstein cases (n=20,40) and achieves only 10% success on the metabolic model. These results confirm that combining global metaheuristic search with high-order local refinement yields a robust, scalable solver for complex biological and engineering nonlinear systems, though performance on problems with dense high-dimensional Jacobians may require further adaptation.

Structured Neural Chaos: An Adaptive Surrogate Modeling Framework for Functional Uncertainty Quantification and Global Sensitivity Analysis

Variance-based global sensitivity analysis (GSA) plays a key role in uncertainty quantification by identifying the contributions of uncertain inputs to the variability of the model response. The repeated model evaluations required for these tasks are often prohibitively expensive; surrogate models provide an efficient alternative by constructing inexpensive approximations of the underlying system response. Constructing surrogate models that combine scalability and interpretability for systems with high-dimensional stochastic inputs and functional responses remains challenging, particularly when sensitivity estimates are required across spatial or temporal domains. Polynomial chaos expansion (PCE) provides an effective framework for uncertainty propagation and sensitivity analysis due to its orthogonal structure and direct relationship with variance-based sensitivity measures. However, PCE suffers from the curse of dimensionality, whose computational burden is amplified for problems with functional responses. In this work, we introduce the Structured Neural Chaos (sNC) expansion as a surrogate modeling framework for variance-based GSA, inspired by the interpretability and orthogonal structure of PCE. The proposed framework retains the interpretability of structured decompositions while leveraging the expressive power of neural networks. The sNC expansion mirrors a truncated functional ANOVA decomposition, where each interaction component admits a separable low-rank approximation whose basis functions and coefficients are parameterized by neural networks. The expansion is constructed sequentially, adaptively identifying the dominant modes within each ANOVA subspace and determining the effective complexity of the representation. The resulting structure enables the extraction of statistical and sensitivity quantities directly from the coefficients of the sNC expansion at negligible cost.