This work introduces agentic Bayesian optimization: a paradigm in which an LLM agent is the central decision maker in the BO loop while a Bayesian backend provides the uncertainty-aware optimization substrate, and demonstrates the practical value of agentic BO in dynamic settings.
Abstract
Bayesian optimization (BO) has become the standard tool for sample-efficient optimization and owes its efficiency to uncertainty-aware search driven by generic statistical priors. Richer domain priors can improve BO in principle, but encoding them through tailored kernels or problem structure is difficult and rarely done in practice. LLMs can help sidestep this difficulty by making informal priors from natural language, code, and documentation directly available to the optimizer. However, existing LLM-based BO methods either insert the LLM into a fixed role (surrogate, acquisition proxy, or configuration interface) or hand it broad control, sacrificing the systematic exploration that makes BO reliable. We introduce agentic Bayesian optimization: a paradigm in which an LLM agent is the central decision maker in the BO loop while a Bayesian backend provides the uncertainty-aware optimization substrate. The agent configures the problem, queries the backend, selects and commits evaluations, and can revise the optimization strategy during the run by tightening bounds, switching acquisition functions, proposing targeted evaluations, or even reframing the problem following new instructions or observed evidence. We instantiate this idea in Sara, a surrogate-augmented autoresearch agent, and lenz, a modular BoTorch-based backend that the agent can inspect and modify through a structured interface. Across synthetic and real-world benchmarks, Sara preserves the reliability of state-of-the-art BO without prior knowledge, outperforms LLM-based baselines, and uses natural-language priors to improve beyond standard BO. We further demonstrate the practical value of agentic BO in dynamic settings, where Sara reconfigures the full optimization problem on the fly as requirements change, a capability not previously available in standard BO.
A unified framework, KENDO (Kernel ENsemble Disagreement-aware Operator), is proposed that integrates Ensemble Gaussian Processes (EGP) with disagreement-aware acquisition strategies and extends the approach to multi-objective optimization via random scalarization that preserves the single-optimizer conditioning structure.
Heng Zhang, Haotian Xiang, Qin Lu et al.· 0 citations
Bayesian optimization (BO) relies on a surrogate model and an acquisition function, yet the most suitable choices vary across tasks and optimization stages. Automated Bayesian optimization (AutoBO) addresses this variability by adapting BO components online. However, existing AutoBO methods either adapt one component, leaving the other mismatched and creating a bottleneck, or jointly select surrogate--acquisition pairs under a shared criterion, overlooking their distinct roles: surrogate selection depends on predictive reliability, whereas acquisition adaptation should respond to campaign context.In this paper, we propose DASH, a Decoupled Adaptive Surrogate--Acquisition Harness for large-language- model (LLM)-enhanced AutoBO. DASH selects surrogates by predictive reliability, uncertainty calibration, and ranking consistency; its two-stage acquisition controller periodically reallocates quotas across acquisition functions, builds a BO shortlist accordingly, and delegates final selection to an LLM. DASH also incorporates an integrated harness, consisting of knowledge-guided warm start and structured memory, to ground optimization in domain knowledge and accumulated feedback. Across four chemical optimization tasks, DASH outperforms the best AutoBO baseline by 12.51% in trajectory-level Acceleration Factor and 5.00% in endpoint Enhancement Factor. Results remain strong across LLM backbones, and ablations verify the complementary contributions of all components. Full-table and behavioral contamination checks find no detectable evidence that direct benchmark memorization or source-cell leakage explains these gains.
Changquan Zhao, Yuxiang Sun, Ruihao Zhu et al.· 0 citations
BOCoDe is introduced, an open-source, PyTorch-native benchmark comprising 307 BBO problems, including 159 engineering design tasks and widely used synthetic and HPO benchmarks that establish a reproducible and extensible foundation for developing and evaluating BO methods that better reflect the demands of engineering design.
Rosen Yu, Christophe Hatterer, A. Narayanan et al.· 0 citations
Bayesian optimization is increasingly used to guide data-efficient experimentation in chemistry, materials science, and related laboratory settings, but its practical performance depends strongly on how well surrogate-model assumptions match the geometry and noise structure of the underlying objective. We introduce tidyHEBO, a robust Bayesian optimization model inspired by heteroskedastic evolutionary Bayesian optimization (HEBO) for single-objective, sequential optimization. tidyHEBO reconstructs the HEBO design philosophy in BoTorch and revises surrogate training, output-warping selection, acquisition function evaluation, and Pareto-front search. We benchmarked tidyHEBO on synthetic functions, Olympus emulators, fully experimental reaction-optimization datasets, needle-in-a-haystack (NIAH) materials problems, and Bayesmark hyperparameter optimization tasks. On these tasks tidyHEBO achieved competitive to superior performance and improvement in robustness across repeated optimization runs. We therefore propose tidyHEBO as a practical tool for sequential experimentations and a strong general-purpose benchmark for future Bayesian optimization research.
A surrogate-recommendation framework is introduced that predicts the most suitable BO surrogate from inexpensive dataset characteristics and establishes FruBO as a reproducible, compute-aware baseline for Bayesian Optimization and provides practical guidance for surrogate selection under limited computational and experimental budgets.
P. Krokidas, C. Rekatsinas, Vassilis Sioros et al.· 1 citation
Many real‐world scientific and industrial applications require the optimization of expensive black‐box functions. Bayesian optimization (BO) provides an effective framework but often struggles with local optima and lacks interpretability. This paper introduces reasoning BO, a novel framework leveraging reasoning models to guide BO sampling while incorporating multi‐agent systems and knowledge graphs for online knowledge accumulation. We evaluate our approach across 10 diverse tasks, including synthetic functions and complex real‐world chemical optimizations. Reasoning BO progressively refines sampling strategies through real‐time insights and hypothesis evolution, identifying high‐performing regions effectively. In the direct arylation task, our method significantly outperformed traditional BO, increasing yield from 25.20% to 60.07%. Furthermore, we demonstrate that smaller LLMs, after post‐training, can achieve performance comparable to larger counterparts. This framework establishes an intelligent cost‐effective optimization system for scientific discovery, combining LLM reasoning with structured knowledge management.
Zhuo Yang, Daolang Wang, Lingli Ge et al.· Materials Genome Engineering...· 0 citations