A surrogate-recommendation framework is introduced that predicts the most suitable BO surrogate from inexpensive dataset characteristics and establishes FruBO as a reproducible, compute-aware baseline for Bayesian Optimization and provides practical guidance for surrogate selection under limited computational and experimental budgets.
Abstract
Bayesian Optimization (BO) is widely adopted for data-efficient optimization in scientific and engineering applications, yet its computational cost is rarely evaluated alongside optimization performance. Here we present a systematic, compute-aware study of BO that evaluates surrogate models along two axes: optimization quality and computational frugality. Across eight benchmark functions and nine real-world datasets spanning materials science, mechanics, robotics, chemistry, and machine learning, we benchmark four surrogate models: Gaussian Processes, Random Forests, NGBoost, and Bayesian Adaptive Spline Surfaces. We show that Gaussian Process-based BO consistently incurs the highest time and memory overhead without delivering superior optimization or sample efficiency. In contrast, scalable alternatives achieve equal or better performance at a fraction of the computational cost. Motivated by these findings, we introduce a surrogate-recommendation framework that predicts the most suitable BO surrogate from inexpensive dataset characteristics. Together, these results establish FruBO as a reproducible, compute-aware baseline for Bayesian Optimization and provide practical guidance for surrogate selection under limited computational and experimental budgets.
Bayesian optimization is increasingly used to guide data-efficient experimentation in chemistry, materials science, and related laboratory settings, but its practical performance depends strongly on how well surrogate-model assumptions match the geometry and noise structure of the underlying objective. We introduce tidyHEBO, a robust Bayesian optimization model inspired by heteroskedastic evolutionary Bayesian optimization (HEBO) for single-objective, sequential optimization. tidyHEBO reconstructs the HEBO design philosophy in BoTorch and revises surrogate training, output-warping selection, acquisition function evaluation, and Pareto-front search. We benchmarked tidyHEBO on synthetic functions, Olympus emulators, fully experimental reaction-optimization datasets, needle-in-a-haystack (NIAH) materials problems, and Bayesmark hyperparameter optimization tasks. On these tasks tidyHEBO achieved competitive to superior performance and improvement in robustness across repeated optimization runs. We therefore propose tidyHEBO as a practical tool for sequential experimentations and a strong general-purpose benchmark for future Bayesian optimization research.
BOCoDe is introduced, an open-source, PyTorch-native benchmark comprising 307 BBO problems, including 159 engineering design tasks and widely used synthetic and HPO benchmarks that establish a reproducible and extensible foundation for developing and evaluating BO methods that better reflect the demands of engineering design.
Rosen Yu, Christophe Hatterer, A. Narayanan et al.· 0 citations
A unified framework, KENDO (Kernel ENsemble Disagreement-aware Operator), is proposed that integrates Ensemble Gaussian Processes (EGP) with disagreement-aware acquisition strategies and extends the approach to multi-objective optimization via random scalarization that preserves the single-optimizer conditioning structure.
Heng Zhang, Haotian Xiang, Qin Lu et al.· 0 citations
This work benchmarks eleven transfer-learning surrogates against four GP methods under an identical selection rule, fidelity budget, and model size, across nine tasks spanning synthetic functions to real chemistry and materials problems, where transfer-learning surrogates reach substantially better solutions using far less computation.
Jaewook Lee, Ethan Errington, Christian D. Lorenz et al.· 0 citations
Black-box optimization is a ubiquitous problem in science and engineering, often dealing with expensive objective functions with cheaper lower-fidelity proxies available. Multi-fidelity Bayesian optimization (MF-BO) is a principled approach to this problem, leveraging correlations across different fidelities when querying the objective. However, for many important MF-BO tasks, the true highest-fidelity function is prohibitively expensive to be part of the optimization loop. Nevertheless, practitioners often have gold standard data (observations of the highest-fidelity function) obtained from previous experiments that might provide information for the current task. For instance, in molecular optimization, chemists often pick the top-$k$ candidate molecules using various computer simulations, and later reveal their true objective function values. In this work, we demonstrate the suboptimality of standard MF-BO algorithms in the real-world scenarios above, even under ideal assumptions. Next, we mitigate this problem by incorporating historical high-fidelity data accompanied by task descriptors---which can be explicitly given or extracted from unstructured metadata. We demonstrate the effectiveness of our methods on synthetic functions, as well as real-world problems in chemistry and hyperparameter optimization.
G. Sutter, Hao Wang, Luis A. Ricardez-Sandoval et al.· 0 citations
This work introduces agentic Bayesian optimization: a paradigm in which an LLM agent is the central decision maker in the BO loop while a Bayesian backend provides the uncertainty-aware optimization substrate, and demonstrates the practical value of agentic BO in dynamic settings.
Paul Brunzema, Louis C. Tiao, Nhat Le et al.· 0 citations