Skip to content
Log in
Preprint

TATG: Tracking-Aware Testing Objective for LLM-based Test Generation

Guancheng Wang Qinghua Xu Lionel C. Briand
Jul 2026 · 0 citations · 34 references
Computer Science

TL;DR

TATG introduces a unified objective representation that captures testing requirements derived from static analysis and dynamic feedback that enables fine-grained tracking of satisfied and unresolved requirements throughout generation.

Abstract

Complex Java methods remain challenging for automated unit test generation because achieving high coverage and fault detection often requires satisfying branch-specific testing requirements that are not directly visible from a focal method. Recent LLM-based approaches, such as KTester, PANTA, and MUTGEN, leverage project context, static analysis, coverage feedback, or mutation guidance. However, they do not explicitly represent and track individual testing requirements across iterations. As a result, generation may repeatedly target satisfied requirements while overlooking unresolved branches and weak assertions. Existing approaches also optimize structural coverage and mutation effectiveness separately. We present TATG, a tracking-aware LLM-based unit test generation approach. TATG introduces a unified objective representation that captures testing requirements derived from static analysis and dynamic feedback. The representation enables fine-grained tracking of satisfied and unresolved requirements throughout generation. TATG further employs a two-stage workflow: structural rounds improve coverage, followed by mutation-guided hardening rounds that strengthen assertions and improve fault detection. We evaluate TATG on 141 complex Java methods, including the 110 KTester subjects and 31 additional challenging methods. Compared with KTester and PANTA, TATG improves line coverage, branch coverage, and mutation score by 22.15, 20.14, and 37.66 percentage points on average. On a selected subset of focal methods, TATG also achieves performance comparable to a proprietary industrial test generation tool while achieving higher line coverage and mutation score.

arXiv Semantic Scholar
View source

Similar papers

Preprint Jul 2026

When Fuzzing Meets Understanding: LLM-Driven Semantic Test Generation for RTL Verification

ChipFuzzer is proposed, a hardware fuzzing framework that leverages the semantic reasoning capabilities of large language models (LLMs) to improve fuzzing effectiveness and improves average condition coverage and bug detection rate over the strongest baseline.

Kun Wang, Cangyuan Li, Kaiyan Chang et al. · 0 citations
Jul 2026

Type-aware LLM-based Test Generation for Python Programs

Test4Py is presented, a novel framework that enhances type correctness in automated test generation for Python by leveraging the program’s call graph to capture richer contextual information about parameters, and introducing a behavior-based type inference mechanism that accurately infers parameter types and constructs valid test inputs.

Runlin Liu, Zhe Zhang, Yunge Hu et al. · 0 citations
Conference Jul 2026

Improving LLM-Based Unit Test Generation Through Root-Cause-Driven Prompt Design

This paper addresses automated unit test generation with large language models (LLMs). LLM-based test generation has not yet attained a quality level sufficient for practical use in industry. Although LLMs often reproduce API syntax faithfully, they frequently disregard semantic usage constraints and execution-environment dependencies, leading to assertion failures, mock-related errors, and reference/resolution errors. A prior failure analysis of Java unit test generation using GPT-4o classified 2980 trials into eight failure patterns and identified three root-cause mechanisms: external context ignorance, internal context ignorance, and a syntax–semantics gap. Building on that analysis, this paper proposes a prompt design comprising three strategies: (1) making the execution state explicit in the generated test, (2) stating semantic constraints explicitly, and (3) injecting environment constraints prior to generation. In contrast to generic techniques such as few-shot learning or chain-of-thought prompting, each proposed strategy is tied to a specific root-cause mechanism, yielding a systematic design in which each rule is explicitly justified by its correspondence to a specific root-cause mechanism. Experiments on 298 methods with five models (GPT-4o, GPT-5, GPT-5.1-Codex, Claude Sonnet 4.5, and Gemini 2.5 Pro) show improved test execution success rates for every model, with absolute gains ranging from 1.1 to 21.1 percentage points (pp). Mock-related errors were reduced by 61.9%–99.2% relative to the baseline prompt, demonstrating effectiveness against the targeted failure patterns. Finally, conditions under which the strategies transfer to other code-generation tasks are discussed, along with limitations on their scope.

Mizuki Yamada, Masahiko Kato, Juichi Takahashi · 0 citations
Aug 2026

An Automated Unit Test Generation Method Based on Dynamic Validation and Minimal Target Repair Iterative Optimization

Experimental results indicate that the dynamic validation mechanism and minimal target repair strategy can reduce invalid generation while improving the executability, assertion effectiveness, and fault-revealing capability of generated tests.

Xiaorong Zhang, Na Wang, Chunyang Su et al. · 0 citations