Jul 2026· GECCO Companion· pp. 237-240· 0 citations· 14 references
Computer Science
TL;DR
To address clifflike fitness landscapes, a dense reward shaping function providing partial credit for structure, number usage, and numeric closeness is introduced to address clifflike fitness landscapes.
Abstract
We study zeroth-order post-training of language models by optimizing only LoRA adapter parameters with population-based black-box optimizers. Representing each candidate as a single LoRA weight vector with frozen base weights, we compare Differential Evolution (DE) and adaptive variants (JADE, SHADE), Particle Swarm Optimization (PSO), Grey Wolf Optimization (GWO), and Whale Optimization Algorithm (WOA) under a unified pipeline on the Countdown arithmetic task. To address clifflike fitness landscapes, we introduce a dense reward shaping function providing partial credit for structure, number usage, and numeric closeness. Across small models, these optimizers substantially improve reward and accuracy, with adaptive DE variants emerging as consistently strong performers.
CoPES is introduced, a cooperative coevolutionary method that decomposes the full parameter space into lower-dimensional subspaces and searches over them cooperatively to improve optimization efficiency and demonstrate an improved trade-off between memory requirements and training time for agentic LLM post-training und...
Zhiyuan Wang, Sheng-Cai Liu, Jiahao Wu et al.· 2 citations
Evolutionary Soups, a mixture-of-experts framework for fine-grained generation control, with gating networks trained via an evolutionary algorithm, achieves the best hypervolume, linear utility, and Tchebyshev utility among controllable methods on all tasks.
Lingxiao Kong, Steffen Staab, Cong Yang et al.· 0 citations
This paper investigates SoRA algorithm's performance in large-scale models, and derives an improved learning rate configuration and proposes SoRA+, an enhanced variant that introduces a hyperparameter to adjust learning rates proportionally and mitigates the suboptimality of the original SoRA method.
Experimental results demonstrate that LLMDE achieves competitive performance on the benchmark suite while continuously generating high-quality solutions for complex constrained optimization tasks, and successfully demonstrate the viability of embedding LLMs within metaheuristics, paving a promising path toward the desi...
Rong-Mu Chai, V. Snášel, Xiao-Peng Wang et al.· 0 citations
This survey organizes recent optimizers and training optimization methods along four largely independent axes: temporal estimation, update geometry, horizon management, and representation and systems.
This paper presents a search-based testing framework for LLM-based applications that incorporates failure diversity as an explicit optimization objective and maintains an archive of generated tests and rewards distance from that archive, while a repopulation operator periodically replaces non-failing tests in the popul...
Lev Sorokin, Ivan Vasilev, Ken E. Friedl et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
