EvoLLM: High-Coverage, Readable, and Token-Efficient Unit Test Generation for Industrial Java Codebases

Abstract

Adopting automated Java unit-test generation in industrial settings requires high coverage, readability, token efficiency, and modern-Java deployability. No existing technique meets all four: search-based test generators struggle with readability and modern-Java deployability, LLM-centric generators consume tens of millions of tokens, and prior LLM-based test refactoring methods drop baseline coverage by 8–21 percentage points (pp) in our controlled setups. We report on EvoLLM, a hybrid Java unit-test generator we built and deployed to address all four requirements. EvoLLM combines EvoSuiteng—a failure-analysis-driven hardening of EvoSuite that resolves deployability issues and strengthens baseline coverage—with cluster-wise LLM-based refactoring that groups tests by coverage goals and rewrites each cluster in a single LLM call grounded in compact program-under-test context. Across six open-source and three industrial projects, EvoLLM outperforms six state-of-the-art baselines, improving line coverage by 8.2–20.7 pp, branch coverage by 6.2–19.4 pp, and mutation score by 6.8–24.5 pp, while reducing LLM token usage by 47.3%–93.0%. In addition, the test generation stage of EvoLLM demonstrates robust deployability across target projects built with JDKs 8, 11, and 17, while achieving strong effectiveness on a Spring-core release requiring JDK 17. Finally, a 14-participant developer survey further suggests that EvoLLM-generated tests ranked higher in readability than those from prior tools and developer-written tests.