Jul 2026· Proceedings of the Genetic and Evolutionary Computation Conference Companion· pp. 1416-1424· 0 citations· 21 references
TL;DR
Findings indicate that while LLM-based performance prediction is not yet a reliable substitute for benchmarking, it shows potential as a complementary pre-screening tool in iterative algorithm design, particularly in settings where code modifications follow predictable patterns.
Abstract
This study investigates whether the current generation of frontier Large Language Models can predict the relative performance of metaheuristic optimization algorithms from source code alone, potentially reducing the computational cost of benchmarking during automated algorithm design. We task three models (GPT 5.4 Thinking, Gemini 3.1 Pro, and Claude Opus 4.6) with predicting improvement or regression between consecutive iterations of iteratively refined differential evolution, and with ranking independently designed algorithms on standard benchmarks. Initial results are promising (88% accuracy), but accuracy collapses on a second algorithm sequence (22–44%). A controlled comparison experiment provides evidence that the models' predictions correlate with code complexity metrics rather than with actual algorithmic performance. Self-reported confidence scores are poorly calibrated across all models tested. These findings indicate that while LLM-based performance prediction is not yet a reliable substitute for benchmarking, it shows potential as a complementary pre-screening tool in iterative algorithm design, particularly in settings where code modifications follow predictable patterns.
It is found that neutral framing of behavioural features, reporting the feature value without prescriptive advice, consistently outperforms the prescriptive variants, and that prescriptive feedback steers the median feature value in the advised direction in only 37% of cases despite empirically grounded advice.
Symbolic regression (SR) is commonly benchmarked using relative Pareto analysis, where an algorithm is evaluated according to whether it dominates the other methods included in the comparison. While useful, this perspective does not reveal the true attainable limits of short, interpretable expressions, and conclusions may change depending on which competing methods are selected. In this work, we advocate benchmarking with absolute Pareto-optimal (APO) fronts instead. We construct APO fronts for 34 datasets from SRBench by exhaustively searching over short symbolic expressions and by fitting numerical constants with eight commonly used local optimization methods. The resulting fronts provide dataset-specific baselines for the best accuracy-complexity trade-offs attainable within a fixed primitive set. Comparing against the SR methods reported in SRBench shows that many current algorithms remain far from these fronts, especially in the regime of short expressions. We also find that the final fronts are relatively stable across numerical optimizers, suggesting that structural search is a more important bottleneck than constant fitting. These APO fronts provide a reusable benchmarking asset for the evolutionary computation community and a more stable reference point for measuring progress in SR.
Kei Sen Fong, M. Motani· Proceedings of the Genetic a...· 0 citations
Large Language Models (LLMs) are opening new directions for automated heuristic design (AHD), allowing evolutionary methods to create and enhance heuristics for constrained optimization problems (COPs). However, most existing approaches face the challenge of the exploration-exploitation balance, where the evolution needs to escape convergence to homogeneous populations and discover as large a heuristic landscape as possible. To address this challenge, we introduce Quality-Diversity Evolution (QDEvo), a multi-objective framework that integrates Quality-Diversity optimization with LLM-based AHD. At its core, QDEvo employs a semantic survival selection mechanism that clusters algorithms by functional similarity, then applies local Pareto competition. Evaluation on well-known COPs benchmarks and real-world problems shows that our method consistently outperforms the state-of-the-art baseline in both Hypervolume and Inverted Generational Distance metrics. These results facilitate further exploration of the algorithmic design space, while ensuring competitive solution quality and efficiency.
Nam Do Khanh, Nhat Nguyen Tran Minh, Dat Pham Vu Tuan et al.· Proceedings of the Genetic a...· 1 citation
Combinatorial problems appear in numerous industrial applications. A common approach is to formulate these problems as declarative constraint models that can subsequently be compiled to and solved by a range of back-end solvers. Recent work shows that Large Language Models (LLMs) can produce correct models from natural language, but even a correct model can be expensive to solve because performance remains sensitive to modelling choices. In this work, we investigate whether LLMs can automate performance-oriented model reformulation. Inspired by Automatic Heuristic Design (AHD), we use an evolutionary framework in which an LLM proposes candidate reformulations that are verified and benchmarked against the user-defined baseline model. We compare AHD-adapted search strategies that control which prior attempts, instructions, and measured feedback enter each prompt. Existing retention strategies prioritize recency or performance, but do not explicitly diversify the context. To cover this gap, we introduce Profile-Diverse Retention (PDR), which applies Maximal Marginal Relevance (MMR) to instance-level runtime vectors to retain behaviourally diverse attempts. We systematically evaluate the strategies on eight CSPLib problems using validation-based final model selection. The results show that: (i) iterative reformulation can produce substantial held-out speedups; (ii) strategies that keep the retained context diverse outperform those that retain only recent or the fastest attempts; and (iii) validation-based selection improves the held-out speedup of every strategy.
Kostis Michailidis, Dimos Tsouros, Dang Nguyen et al.· 0 citations
This paper presents our submission to the competition on LLM-designed evolutionary algorithms for the GNBG benchmark suite. We study a simple alternative to recent automated algorithm-design frameworks: instead of placing the language model inside a specialized evolutionary loop, we treat optimizer construction as an ordinary coding task. In our setup, a general-purpose coding agent iteratively modifies and evaluates a candidate optimizer inside a small repository that provides a fixed interface, benchmarking harness, and lightweight experiment log. The main question is whether such an agent, without any bespoke search controller, can still discover competitive metaheuristics. Our results indicate that the answer is positive: the resulting optimizer performs well on a substantial subset of GNBG II. These findings suggest that recent progress in language models and tool use may already be sufficient to support practical automated metaheuristic design.
Jan Iłowski, Marcin Małek, Wojciech Achtelik et al.· Proceedings of the Genetic a...· 0 citations
This paper introduces the bi-objective problem class CLIMB and analyzes the runtime of GSEMO and the widely used NSGA-II on this problem, and proves that GSEMO and NSGA-II-DYN, a version of NSGA-II with dynamic population sizes, can find the Pareto front of CLIMB in expected fitness evaluations.