Evolutionary Algorithms for Lightweight Post-Training of Language Models

Abstract

We study zeroth-order post-training of language models by optimizing only LoRA adapter parameters with population-based black-box optimizers. Representing each candidate as a single LoRA weight vector with frozen base weights, we compare Differential Evolution (DE) and adaptive variants (JADE, SHADE), Particle Swarm Optimization (PSO), Grey Wolf Optimization (GWO), and Whale Optimization Algorithm (WOA) under a unified pipeline on the Countdown arithmetic task. To address clifflike fitness landscapes, we introduce a dense reward shaping function providing partial credit for structure, number usage, and numeric closeness. Across small models, these optimizers substantially improve reward and accuracy, with adaptive DE variants emerging as consistently strong performers.