Analyzing search behavior of population-based optimization algorithms using graph neural networks

On the Edit Path to GNN Decisions

Semantic Space Search Trajectory Networks

Search Trajectory Networks (STNs) are a graph-based tool for visualizing and characterizing the behavior of optimization algorithms. STNs'reliance on discretization of the search space has largely confined them to low-dimensional or combinatorial settings. We introduce a methodology for constructing STNs in semantic spaces, defined as the space of a model's predictions on a fixed sample set. Our approach discretizes semantic vectors and aggregates them into network nodes via agglomerative clustering with complete linkage under a normalized Hamming distance. Since any predictor can be summarized by its semantic vector, this method enables comparison of learning dynamics across otherwise incomparable algorithm families. We apply semantic space STNs to classification and regression tasks solved using different machine learning algorithms, recovering known qualitative differences between them. Additionally, we use semantic space STNs to study neural network generalization by contrasting standard training with the label randomization regime of Zhang et al. (2017). The resulting STNs exhibit consistent structural differences, training on real labels produces denser, more efficient and more centralized graphs than training on shuffled labels. Together, our results show that semantic space STNs capture functional training dynamics arising from the interaction between learning algorithms and data, providing a tool for analyzing and comparing learning dynamics across machine learning models and training regimes.

An Ensemble Learning Approach to Graph Learning Based on Evolutionary Graph Neural Architecture Search.

The graph neural networks (GNNs) have been successfully applied to non-Euclidean graph data mining tasks, attracting widespread attention. At present, to achieve promising performance, many researchers use neural architecture search (NAS) on GNNs to enhance applicability and accuracy without requiring significant manual intervention or rich domain knowledge. However, most existing graph NAS (GNAS) algorithms struggle with high-dimensional data and often overlook the characteristics and distribution of different solutions within the search space, which may result in local optima and underutilization of the search potential. This article proposes Ensemble-GNAS, a novel framework that integrates evolutionary GNAS with ensemble learning to address these limitations. Within this framework, we develop two key strategies: an evolutionary GNAS method based on a niche strategy to enhance local search capability and the diversify candidate networks and an ensemble fusion strategy based on a tree-structured Parzen estimator (TPE) to optimize the weights of candidate networks in an ensemble model. The proposed approach is evaluated on: 1) node classification datasets, including citation networks for paper classification and cancer-specific gene interaction networks for driver gene identification and 2) link prediction and graph classification datasets. The experimental results indicate that the proposed approach outperforms the state-of-the-art methods in both classification accuracy and performance stability. Moreover, the ablation study verifies the contribution of the integrated niche and TPE strategies within the overall framework. The source code of Ensemble-GNAS can be obtained from https://github.com/WilfongGuo/EnsembleGNAS.

Surrogate-Assisted Linear Genetic Programming for Evolving Graph Neural Networks for Node Classification

A pairwise surrogate model is introduced that predicts whether an offspring architecture is likely to outperform its parent using genotype-derived features and early training signals, enabling full evaluation only for promising candidates within the evolutionary loop.

A Comprehensive Survey on Symbolic Regression: State-of-the-Art Approaches, Key Applications, Benchmark Evaluations, and Future Research Directions

Symbolic regression autonomously discovers interpretable mathematical expressions from data by optimizing functional structure and associated parameters. Unlike traditional regression approaches that rely on predefined model forms, symbolic regression generates explicit models that maintain high predictive accuracy while providing valuable scientific insights. This survey comprehensively analyzes contemporary symbolic regression methodologies by systematically integrating four paradigmatic approaches: deterministic methods, metaheuristic algorithms, neural-symbolic frameworks, and emerging hybrid strategies. We establish a unified taxonomic framework that bridges evolutionary computation, mathematical programming, and deep learning paradigms. Our analysis reveals convergence patterns toward physics-informed discovery, multi-objective optimization, and human-collaborative frameworks. WWe examine hybrid integration strategies, semantic-aware operators, and constraint-handling mechanisms, and we critically assess evaluation methodologies, benchmarking practices, applications across scientific and engineering domains, and fundamental limitations, including search complexity, overfitting, and high-dimensional scaling challenges. This work makes four key contributions. First, we provide a unified framework analyzing over 300 methods across four paradigms. Second, we offer comprehensive coverage of developments from 2020 to 2025, including transformer-based and large language model approaches. Third, we present a systematic analysis of hybrid strategies and convergence patterns. Fourth, we provide actionable guidance for method selection. This survey establishes a comprehensive contemporary reference for symbolic regression research, highlighting pathways toward robust, interpretable, and scalable automated scientific discovery systems.

GAE: Graph-Augmented Evolution for Scientific Discovery via Reinforcement Optimization

Evolutionary program search guided by Large Language Models (LLMs) has emerged as a powerful paradigm for automated scientific discovery. However, current approaches are fundamentally constrained by three bottlenecks: structurally blind parent selection, sparse whole-program evaluation rewards, and static mutation operators that fail to adapt during search. We present GAE (Graph-Augmented Evolution), a framework that resolves these limitations through a tightly coupled, three-pillar architecture. First, a relational graph neural network (GNN) parses programs into typed computation graphs, producing structure-aware embeddings. Second, an RL-optimized meta-controller leverages these embeddings to replace blind evolutionary sampling with a directed policy, dynamically selecting optimal parents and mutation directions based on reward history. Third, an online GRPO fine-tuning loop continuously updates the LLM mutation operator at test-time using group-normalized evaluation rewards, directly aligning the model's generation distribution with high-fitness structural edits. We evaluate GAE on a challenging scientific discovery task: symbolic regression for complex nonlinear oscillator systems. By transforming stochastic search into a directed, self-improving trajectory, GAE efficiently discovers closed-form physical equations, consistently matching or outperforming static LLM-driven baselines and achieving state-of-the-art out-of-distribution performance.