GENE: Generalizable Need-Driven Evolution of Coding Agents with Post-Evolution Verification
Abstract
Agent harnesses critically affect coding-agent performance, but their automatic evolution can overfit individual failures and introduce unforeseen regressions. We present GENE, a training-free framework for generalizable and verifiable harness evolution. Instead of translating each task failure directly into a harness edit, GENE derives evidence-linked component needs from execution profiles, refined trajectories, and task reflections, and assesses whether each need represents a reusable capability. An Evolve Agent aggregates this evidence to modify heterogeneous harness components, while an independent Judge performs pre-application review and subsequent task outcomes provide post-evolution regression evidence. On Terminal-Bench 2.1, GENE matches AHE’s best evolution-set pass@1 of 56.7%. On the 59-task holdout, its selected checkpoint passes 27 tasks versus 26 for AHE and the seed; two-sided exact McNemar tests detect no difference (both $\boldsymbol{p}=\mathbf{1 . 0 0}$). Across adjacent checkpoints, descriptive outcomes show 73.5% versus 61.3% pass retention and 8 versus 43 new exception transitions. The audit records four Judge-requested revisions but also undetected damage to tracing, motivating controlled evolution with protected invariants. The source code is publicly available at https://github.com/BillChou5555/GENE.