A closed-loop evolutionary algorithm that guides an LLM to generate complete, executable PINN configurations across generations, using measured training outcomes to determine subsequent search decisions to demonstrate the feasibility of evolutionary-algorithm-guided LLMs for PINN design on a controlled PDE while motivating broader, physics-aware evaluation.
Abstract
Physics-informed neural networks (PINNs) are unusually sensitive to interacting choices of architecture, activation, loss weighting, collocation, optimization, and constraint enforcement. Large language models (LLMs) can propose these choices, but independent recommendations do not accumulate experience from previously trained PINNs. We propose a closed-loop evolutionary algorithm that guides an LLM to generate complete, executable PINN configurations across generations, using measured training outcomes to determine subsequent search decisions. The algorithm maintains an evaluated population and lineage, applies parent-conditioned mutation and crossover, preserves elite and diverse solutions, rejects effective duplicates, and converts parent-relative successes and failures into the next-generation context supplied to the LLM. Every proposed configuration is executed directly under an exact optimizer-step budget. On a one-dimensional multiscale wave equation, two independent ten-generation runs trained 60 PINNs for 600,000 optimizer steps. In both runs, the best configuration appeared in the final generation, with best mean-squared error reduced by 2.97\% and 95.38\% relative to the initial population. The stronger run validated residual connections and increased depth on separate branches, combined them in a later generation, and then refined width and collocation density. It also revealed that low solution error can coexist with a high PDE residual. These results demonstrate the feasibility of evolutionary-algorithm-guided LLMs for PINN design on a controlled PDE while motivating broader, physics-aware evaluation.
Physics-informed neural networks (PINNs) have emerged as a powerful paradigm for solving partial differential equations (PDEs), yet their performance heavily relies on the manual, trial-and-error engineering of neural representations, loss formulations, and optimization dynamics. While Large Language Models (LLMs) offer a promising avenue for automated design, unconstrained code generation often yields mathematically invalid or numerically unstable solutions under strict scientific computing constraints. To bridge this gap, we propose \textbf{EvoPINN}, an agentic framework that reformulates PINN development from labor-intensive manual design into a rigorous, execution-grounded algorithm discovery problem. EvoPINN navigates a modular search space by decoupling neural representations from training programs, utilizing an LLM agent to iteratively propose memory-conditioned programmatic modifications. To ensure scientific validity, all candidates undergo strict structural verification and budget-matched PDE evaluation. Extensive experiments across diverse PDE regimes (oscillatory, elliptic, dissipative, and nonlinear transport) demonstrate that EvoPINN discovers PDE-specialized learning algorithms that significantly reduce relative $L_{2}$ error compared to baselines. Crucially, EvoPINN autonomously invented SLRC-PINN, a novel architecture whose performance gains persist under rigorous parameter-matched comparisons, establishing the viability of execution-grounded agents for discovering genuinely new scientific computing mechanisms.
LLM-driven evolutionary search can discover algorithm designs that achieve Pareto-efficient trade-offs difficult to reach through manual design, with SMAC hyperparameter optimization integrated into the evolutionary loop.
G. Laskaris, R. Brasher, Niki van Stein et al.· 0 citations
Physics-informed neural networks (PINNs) are commonly evaluated via a single aggregate residual, assuming a smaller residual indicates a better solution. Testing this directly across three constrained PDE systems, I find this assumption can systematically fail. In matched pairs of solvers differing only in whether a defining structural identity is hard-wired or penalized, the penalized variant frequently attains a lower equation residual while violating that identity by several orders of magnitude, causing the exact variant to be falsely ranked worse. Over 64 matched pairs spanning two systems, four network variants, and eight seeds, this inversion occurs in 83\% of cases (95\% Wilson CI: 72--90\%), with rates from 72\% to 94\% across systems. Testing across six architectures--MLP, cPINN, XPINN, hp-VPINN, and physics-informed DeepONet and FNO--inverts the ranking in 46 of 48 pairs, indicating that this variability is problem-dependent rather than specific to the approximator. A third, larger vorticity--streamfunction problem shows the same ordering: the residual-optimal solver violates its structural identity by over six orders above tolerance, despite a residual margin of only 9.37%. Because a scalar loss cannot expose this, I introduce a lexicographic admissibility gate spanning the structural identity, boundary trace, and a solvability integral that must vanish independently of the equation residual. All three must pass before residuals can compete. This gate catches three artifact classes but misses a fourth: a prescribed-structure prior yields fields that pass every single-run check, yet deleting the source term reveals that 97\% of the reported structure survives removal of the physics. Reference data and figures accompany the paper.
Evolutionary program search guided by Large Language Models (LLMs) has emerged as a powerful paradigm for automated scientific discovery. However, current approaches are fundamentally constrained by three bottlenecks: structurally blind parent selection, sparse whole-program evaluation rewards, and static mutation operators that fail to adapt during search. We present GAE (Graph-Augmented Evolution), a framework that resolves these limitations through a tightly coupled, three-pillar architecture. First, a relational graph neural network (GNN) parses programs into typed computation graphs, producing structure-aware embeddings. Second, an RL-optimized meta-controller leverages these embeddings to replace blind evolutionary sampling with a directed policy, dynamically selecting optimal parents and mutation directions based on reward history. Third, an online GRPO fine-tuning loop continuously updates the LLM mutation operator at test-time using group-normalized evaluation rewards, directly aligning the model's generation distribution with high-fitness structural edits. We evaluate GAE on a challenging scientific discovery task: symbolic regression for complex nonlinear oscillator systems. By transforming stochastic search into a directed, self-improving trajectory, GAE efficiently discovers closed-form physical equations, consistently matching or outperforming static LLM-driven baselines and achieving state-of-the-art out-of-distribution performance.
Goal-directed optimization is essential for steering molecular generators to propose candidates with desired properties. However, it is often implemented with policy-gradient reinforcement learning, which requires a generation-trajectory log-probability whose form depends on the model architecture and generation procedure. This makes an optimizer difficult to reuse across architectures and conditional generative designs. Supervised fine-tuning needs none of that machinery, but its update is driven by a fixed dataset, so the reward never enters the update. We introduce Elite-Weighted Supervised Fine-tuning (EW-SFT), which uses reward to guide elite selection of high-scoring molecules, and updates the model by its own pretraining loss on that set. Ablations show that reward information is passed primarily through elite selection, rather than through continuous weighting within the selected set. Because the update consumes only scored molecules and the model's native loss, the same rule applies across autoregressive, masked-diffusion, and discrete-flow generators, and across de novo, motif-extension, and linker-design tasks. Under a fixed budget of 3D shape alignment oracle calls on two kinase reference compounds, EW-SFT consistently outperforms the corresponding native optimizers. It further improves goal-directed optimization under a 2D similarity oracle on four held-out references and achieves comparable performance on a sample-efficiency benchmark without a trajectory-level RL formulation. These results demonstrate that EW-SFT is a unified and effective optimizer across molecular generators, design constraints, references, and oracles.
Shiyun Wa, Yifei Wang, A. G. Green et al.· 0 citations
Physics-informed neural networks (PINNs) approximate partial differential equations (PDEs) by enforcing governing equations and boundary conditions during training, but their accuracy depends on how collocation points are distributed and updated. We propose spatiotemporal compositional active sampling (STCAS), a reference-assisted offline configuration procedure that uses an analytic or high-accuracy numerical solution to rank complete three-stage sampling plans. It screens eight fixed rules, forms a task-specific shortlist, and evaluates bounded fixed, switched, and locally blended plans with independent selection sets and a composition guard. A safety-anchor decision retains the standard PINN unless the selected candidate is at least 5% better. Across five evaluations on 18 analytically specified two-dimensional Poisson tasks, this protocol improves 16 task means and ties two, reducing aggregate relative-L2 error by 12.8% (hierarchical-bootstrap 95% interval [6.78%,19.32%]; one-sided paired Wilcoxon p=2.19×10−4). Against the confirmed fixed plan, aggregate error decreases by 8.2%. In comparison experiments designed for two transfer tasks and matched for main PINN training budgets, STCAS achieves the lowest aggregate mean reported error among the compared methods for both a steady convection–diffusion equation and a nonlinear time-dependent Burgers equation; its offline search cost is additional.
Ju-Zheng Zhang, Shi-Yang Li, Tao Zhu et al.· Mathematics· 0 citations