BOCoDe is introduced, an open-source, PyTorch-native benchmark comprising 307 BBO problems, including 159 engineering design tasks and widely used synthetic and HPO benchmarks that establish a reproducible and extensible foundation for developing and evaluating BO methods that better reflect the demands of engineering design.
Abstract
Bayesian optimization (BO) is a sample-efficient, surrogate-based approach to black-box optimization (BBO), but its evaluation remains dominated by synthetic functions and hyperparameter optimization (HPO) tasks that are typically low-dimensional and single-objective. Engineering design poses a substantially different regime: problems are physics-based, often high-dimensional, constrained by requirements such as cost and manufacturability, and may involve multiple objectives or mixed variables. To close this benchmarking gap, we introduce BOCoDe, an open-source, PyTorch-native benchmark comprising 307 BBO problems, including 159 engineering design tasks and widely used synthetic and HPO benchmarks. Each problem includes cited provenance and machine-readable metadata that supports programmatic discovery, including by LLM-based agents, and all tasks are exposed through a unified API compatible with open-source BO libraries. We evaluate 31 BO and evolutionary algorithms across five problem classes spanning single- and multi-objective optimization, constrained and unconstrained settings, and mixed-variable search spaces. Analyses of problem structure show that engineering tasks uniquely span constrained and multi-objective settings that synthetic and HPO suites rarely cover, while embeddings from a tabular foundation model separate them most clearly from HPO tasks. Algorithm rankings also vary substantially across domains; in several problem classes, rankings obtained on standard benchmarks do not transfer to engineering tasks. BOCoDe establishes a reproducible and extensible foundation for developing and evaluating BO methods that better reflect the demands of engineering design. Code&data can be found at https://github.com/rosenyu304/BOCoDe
A surrogate-recommendation framework is introduced that predicts the most suitable BO surrogate from inexpensive dataset characteristics and establishes FruBO as a reproducible, compute-aware baseline for Bayesian Optimization and provides practical guidance for surrogate selection under limited computational and experimental budgets.
P. Krokidas, C. Rekatsinas, Vassilis Sioros et al.· 1 citation
LLM-driven evolutionary search can discover algorithm designs that achieve Pareto-efficient trade-offs difficult to reach through manual design, with SMAC hyperparameter optimization integrated into the evolutionary loop.
G. Laskaris, R. Brasher, Niki van Stein et al.· 0 citations
Black-box optimization is a ubiquitous problem in science and engineering, often dealing with expensive objective functions with cheaper lower-fidelity proxies available. Multi-fidelity Bayesian optimization (MF-BO) is a principled approach to this problem, leveraging correlations across different fidelities when querying the objective. However, for many important MF-BO tasks, the true highest-fidelity function is prohibitively expensive to be part of the optimization loop. Nevertheless, practitioners often have gold standard data (observations of the highest-fidelity function) obtained from previous experiments that might provide information for the current task. For instance, in molecular optimization, chemists often pick the top-$k$ candidate molecules using various computer simulations, and later reveal their true objective function values. In this work, we demonstrate the suboptimality of standard MF-BO algorithms in the real-world scenarios above, even under ideal assumptions. Next, we mitigate this problem by incorporating historical high-fidelity data accompanied by task descriptors---which can be explicitly given or extracted from unstructured metadata. We demonstrate the effectiveness of our methods on synthetic functions, as well as real-world problems in chemistry and hyperparameter optimization.
G. Sutter, Hao Wang, Luis A. Ricardez-Sandoval et al.· 0 citations
This work presents a surrogate-assisted multi-objective optimization approach that leverages symbolic regression (SR) to construct interpretable analytical approximations of expensive black-box functions. It uses iterative construction of surrogates (symbolic models) and Tchebycheff scalarization along a set of parallel reference vectors to search for well-distributed solutions on the Pareto front. Optimization of the symbolic models is performed using a nonlinear programming (NLP) solver in lieu of heuristic search. A distance-based subset selection strategy is used to select a candidate for true evaluation, ensuring efficient use of limited evaluation budget in a steady-state framework. The approach is compared against a Kriging-assisted NSGA-II, also implemented within a steady-state framework. Experiments are performed on six benchmark problems—covering both constrained and unconstrained cases, including a practical engineering benchmark of bracket design problem. The findings highlight the strengths and limitations of SR-based surrogates combined with NLP, and position them as a viable alternative to conventional surrogate-assisted evolutionary algorithms.
Kannan Sekar, H. Singh, Tapabrata Ray· Proceedings of the Genetic a...· 0 citations
This work introduces agentic Bayesian optimization: a paradigm in which an LLM agent is the central decision maker in the BO loop while a Bayesian backend provides the uncertainty-aware optimization substrate, and demonstrates the practical value of agentic BO in dynamic settings.
Paul Brunzema, Louis C. Tiao, Nhat Le et al.· 0 citations
A generalizable transformation methodology is developed that converts compatible single- or multi-objective sequential Bayesian optimization methods into B-MOBO methods that are as effective as existing methods at solving optimization problems in terms of solution quality and show improvements when real-time is considered.
Adelle Holder, H. Debruin, Jesse M. Sestito· PLoS ONE· 0 citations