Research artifact for Automatic Generation of Unit Test Cases Based on Program Feedback Reinforcement Learning

Abstract

This research artifact accompanies the manuscript Automatic Generation of Unit Test Cases Based on Program Feedback Reinforcement Learning by WeiDong Zhao. It contains reproducibility materials, not the unpublished manuscript full text. The artifact preserves an actually executed three-seed Qwen2.5-Coder-0.5B-Instruct SFT, scalar pairwise reward-learning, and PPO implementation: 3,072 optimizer updates, before/after adapters, 768 matched holdout generations, and 768 reward cross-evaluation decisions. A subsequent 12-method public-library diagnostic records 144 Java generation, compiler, JUnit and JaCoCo attempts. Frozen protocols, configurations, selected public-data identities, source code, raw measurements, statistical scripts, and SHA-256 manifests are included. Natural-code preference validation includes four 1,500-record transformation families and an additional 1,500 invocation-aware focal replacement attempts. Replacement passed the static property-invariance gate in 1,498 cases; identifier deletion passed in 43. These are explicitly constructed synthetic preference negatives. An earlier actually compiled 36-fixture, 108-mutant controlled study is a supplementary counterexample, not a natural-project performance benchmark. Separate archived fixed-model inference and repair records are not RL training. The gated-minus-legacy PPO contrast is 4.17 percentage points, but the prespecified Holm-adjusted test did not establish an advantage (p=0.1562). Passing target-executing candidates in the subsequent executable diagnostic are 0, 3, 3 and 4 out of 36 attempts per checkpoint. The extra success is one seed on one additional method. No historical ChatGLM reproduction, repository-level generalization, or superiority over complete recent systems is claimed. Historical missing training logs have not been reconstructed. The 12 October 2026 release reverified the archived training and executable records, independently recomputed the frozen analyses, and replayed a recorded 16-token checkpoint prefix. It did not rerun all training or Java generation. Large public base weights and full project/build dependencies are identified by upstream revision, file hashes and original local indexes rather than duplicated. The README distinguishes analysis-only commands, copied records, indexed dependencies and original runnable hierarchy. The associated manuscript has not been formally published. The author reports no external funding and no conflict of interest. Author-created research records and documentation are attributed to WeiDong Zhao; third-party source excerpts remain subject to their original copyright and licenses.