DeepPySR is introduced, which addresses SR's challenges with a dynamic variable-pruning schedule to remove irrelevant features during search, an exponential Pareto selection criterion that eliminates trade-offs between accuracy and complexity, and a multi-layer architecture for hierarchical symbolic composition.
Abstract
Symbolic regression (SR) discovers analytical equations from data, yielding glass-box models with directly interpretable formulas, unlike black-box methods that rely on unstable post-hoc tools such as SHAP or LIME. This transparency is crucial in clinical medicine and social science, but SR faces three challenges: high-dimensional inputs, principled selection of Pareto-front formulae, and data irregularities such as multicollinearity and class imbalance. We introduce DeepPySR, which addresses these issues with a dynamic variable-pruning schedule to remove irrelevant features during search, an exponential Pareto selection criterion that eliminates trade-offs between accuracy and complexity, and a multi-layer architecture for hierarchical symbolic composition. On four Feynman physics benchmarks and seven biomedical and social-science datasets, DeepPySR outperforms PySR and baselines on body fat (R$^2$: 0.794 vs.\ 0.702), heart disease (F1: 0.898 vs.\ 0.787), student performance (R$^2$: 0.964 vs.\ 0.948), and Raine BMI (R$^2$: 0.525 vs.\ 0.370), producing interpretable formulas aligned with domain risk factors.
Symbolic regression autonomously discovers interpretable mathematical expressions from data by optimizing functional structure and associated parameters. Unlike traditional regression approaches that rely on predefined model forms, symbolic regression generates explicit models that maintain high predictive accuracy while providing valuable scientific insights. This survey comprehensively analyzes contemporary symbolic regression methodologies by systematically integrating four paradigmatic approaches: deterministic methods, metaheuristic algorithms, neural-symbolic frameworks, and emerging hybrid strategies.
We establish a unified taxonomic framework that bridges evolutionary computation, mathematical programming, and deep learning paradigms. Our analysis reveals convergence patterns toward physics-informed discovery, multi-objective optimization, and human-collaborative frameworks. WWe examine hybrid integration strategies, semantic-aware operators, and constraint-handling mechanisms, and we critically assess evaluation methodologies, benchmarking practices, applications across scientific and engineering domains, and fundamental limitations, including search complexity, overfitting, and high-dimensional scaling challenges.
This work makes four key contributions. First, we provide a unified framework analyzing over 300 methods across four paradigms. Second, we offer comprehensive coverage of developments from 2020 to 2025, including transformer-based and large language model approaches. Third, we present a systematic analysis of hybrid strategies and convergence patterns. Fourth, we provide actionable guidance for method selection. This survey establishes a comprehensive contemporary reference for symbolic regression research, highlighting pathways toward robust, interpretable, and scalable automated scientific discovery systems.
Vikas Palakonda, Samira Ghorbanpour, Sangseok Yun et al.· Archives of Computational Me...· 0 citations
This work presents a multiview symbolic regression method that decomposes predictions into a shared core equation and view-specific residual corrections, and demonstrates that effective multiview symbolic regression depends more on appropriate initialization of the search space than on sophisticated search operators.
H. Lira, Luis Martí, Nayat Sánchez-Pi· Annual Conference on Genetic...· 0 citations
Symbolic regression (SR) is the task of discovering underlying patterns from data and representing them using mathematical expressions. Current machine learning approaches to SR often lack a profound understanding of the intrinsic mathematical and physical principles governing these expressions. While the pioneering AI Feynman method leverages the mathematical properties underlying the data, its expression simplification mechanism suffers from a narrow scope of applicability and is prone to failure on complex equations. Furthermore, its underlying mechanisms rely heavily on brute-force searches for sub-expressions, severely limiting its practical utility. Through rigorous mathematical deduction and proofs, we propose our method, Deep Divide and Reduce in Symbolic Regression (DDRSR). DDRSR fundamentally broadens the applicability of expression decomposition and reduction, circumvents the need for brute-force sub-structure searches, and ensures both wider versatility and strict theoretical correctness. Empirical evaluations demonstrate that these theoretical principles yield significant advantages in both expression decomposition and numerical regression tasks. Finally, we discuss the applicable scenarios and inherent limitations of this paradigm, alongside promising directions for future research.
Symbolic Regression (SR) aims to discover analytical equations from observational data and plays a central role in scientific modeling. While recent Large Language Model (LLM) based approaches show promise, they face two limitations. First, they lack data analysis mechanisms for uncovering variable dependencies, which reduces the efficiency of equation discovery. Second, most methods rely on single-objective evaluation focused solely on fitting error. This neglect of structural complexity and generalization often causes models to converge prematurely to local optima, limiting their ability to explore the broader equation space. We propose Multi-Objective Tool-augmented Symbolic Regression (MOT-SR), a unified framework that integrates external analytical tools to extract structural priors and guide equation generation, while jointly optimizing for accuracy, complexity, and generalization via a multi-objective evaluation module that maintains a dynamic Pareto front. MOT-SR employs two collaborative LLM modules: a Meta Strategy Generator, which selects tools and synthesizes structural optimization strategies based on Pareto-optimal equations, and an Equation Generator, which produces new candidate equations accordingly. The system operates in a closed-loop manner, continuously refining both strategies and equation structures. Across 40 standard tasks, MOT-SR outperforms existing SR methods in accuracy, generalization, and efficiency. We further validate MOT-SR on extreme mass-ratio inspiral (EMRI) orbital modeling, an important problem in space-based gravitational-wave astronomy where small local errors can accumulate substantially over long-term evolution. The discovered interpretable correction achieves the lowest trajectory-level integration error on held-out configurations. These results demonstrate the potential of MOT-SR to enable reliable modeling of long-horizon scientific dynamics.
Boxiao Wang, Runxian Wang, Kai Li et al.· 0 citations
Symbolic Regression (SR) is a core challenge in both physics and artificial intelligence, aiming to identify mathematical equations from experimental data. Recent impressive advancements in Large Language Model (LLM)-based SR methods have addressed the limitations of traditional approaches in flexibly incorporating prior knowledge to enhance accuracy, but they still face challenges such as high costs and scalability with numerous variables. To overcome these issues, we introduce SymBOL. This general-purpose symbolic learning framework uses Bayesian Optimization (BO) to guide the generation of high-quality mathematical expressions from the LLM while accommodating complex tasks with a large number of interdependent variables. Experiments on benchmark datasets demonstrate that SymBOL significantly outperforms baseline methods in both accuracy and efficiency. Notably, compared to advanced LLM-based approaches, SymBOL achieves a 24.85% improvement in average accuracy while reducing computational costs by 28.73%. This advantage extends to high-dimensional SR tasks, where SymBOL substantially lowers the average error. Furthermore, when applied to real-world systems in materials science and epidemiology, SymBOL accurately recovers governing equations and provides interpretable pathways for equation discovery. These findings underscore the potential of SymBOL for advancing scientific discovery.
Jiaxu Cui, Qifei Li, Weiting Liu et al.· IEEE Transactions on Pattern...· 0 citations
Scientific equation discovery must combine broad domain priors with strict numerical testing. Symbolic regression supplies numerical grounding but faces a combinatorial search space, whereas many language-model systems ask the model to propose or select formulas directly. We test a different division of labour. We compare role specifications in which the language model acts as equation author, candidate decider or search controller, alongside end-to-end language-model and purely numerical baselines. In the controller setting we propose here, implemented as LLM-PySR, language models specify variables, operators, transformations and search depth; symbolic regression enumerates and fits expressions; and deterministic metrics govern retention. Across 74 AI-Feynman equations and seven complex formula-recovery tasks, search control achieved the strongest observed balance of accuracy, complexity, stability and cost. On an independent battery dataset, LLM-PySR identified a compact piecewise-linear relation between early voltage-curve displacement and cycle life. The results suggest that language models should shape hypothesis exploration rather than decide which equations survive.