Skip to content
Preprint

Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments

Jul 2026 · 0 citations · 51 references
Computer Science

TL;DR

This work presents Change2Task, a system grounded in repository history that converts merged pull requests into verified tasks on healthy modern revisions of the same repository, and provides executable data for coding agent training and evaluation while reducing repeated environment setup, storage, and task construction effort.

Abstract

Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation. Each task must couple a realistic software state with a specification, development tools, and reliable verification. To expand this supply, we present Change2Task, a system grounded in repository history that converts merged pull requests into verified tasks on healthy modern revisions of the same repository. It aligns historical evidence with evolved code, reconstructs task states through Patch Reversal, Code Mapping, or Agent Reconstruction, and validates the lifecycle from a healthy base to a task state and a restored state. By deriving multiple tasks grounded in developer evidence from maintained environments, Change2Task provides executable data for coding agent training and evaluation while reducing repeated environment setup, storage, and task construction effort. We evaluate the system through five common and widely adopted coding agent task families: Bug Fix, Feature Addition, Test Generation, Application Programming Interface Migration, and Security Repair. Starting from 1,130 source changes eligible for construction, Change2Task achieves 79.6% verified task construction success across these task families. On a matched candidate set, it recovers 29.2% more verified tasks than a construction baseline based on pull requests. Historical and reconstructed cases achieve up to 98.0% matched outcome agreement under agent evaluation, while reuse of modern bases reduces measured expenditure across the complete pipeline by 10.8%.

View source

Similar papers

Preprint Jul 2026

ICAE-Bench: Evaluating Coding Agents as Interactive Project Builders

ICAE-Bench, a benchmark for evaluating coding agents under interactive project-building settings, starts from a fuzzy product requirement, simulating the dynamic paradigm with an automated User Agent, and introduces three key designs.

Zhongyuan Peng, Dan Huang, Chuyu Zhang et al. · 1 citation
Preprint Aug 2026

A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents

SWE-RPG is introduced, a repository-level benchmark that combines executable patch evaluation with validated ground-truth references (GTs) for Requirement Clarification and Implementation Planning, and suggests implicit-requirement recovery as a key candidate direction for improving coding agents.

Xin Zhou, C. Chong, Kisub Kim et al. · 0 citations
Review Aug 2026

Self-Evolving Coding Agents

This survey aims to clarify the conceptual boundaries of self-evolving coding agents and provide a foundation for designing more adaptive, reliable, and software-aware agentic systems.

H. Zhou, Haichuan Hu, Ye Shang et al. · 0 citations
Preprint Aug 2026

Evaluating Agentic Code Repair Capabilities in Distributed Systems

DDBench is introduced, a code-repair benchmark of 60 historical bugs mined from 13 open-source distributed systems, partitioned into three difficulty tiers, isolating the effect of debugging context from model capability.

Yibo Yan, Huijuan Wang, Junzhou He et al. · 0 citations
Preprint Jul 2026

Preventing Premature Commitment in Coding Agents with an Evidence-Conditioned Execution Layer

ECLoop is presented, an execution layer that interposes between the agent and the repository to enforce evidence-conditioned execution and shows that each of ECLoop's three operations contributes distinct value and that structured evidence conditions outperform an equivalent natural-language summary.

Yisen Xu, Chenglin Li, Zehao Wang et al. · 1 citation
Preprint Jul 2026

LoopsBench: From Harness Engineering to Loop Engineering in Coding Agent Evaluation

Coding agent infrastructure is shifting from harness engineering toward loop engineering as coding agents are deployed for sustained long-horizon software development. Existing benchmarks often center on localized tasks or end-state outcomes, offering limited insight into sustained execution. We introduce LOOPSBENCH, a long-horizon benchmark for loop engineering in coding agent evaluation. Each task is a dependency DAG over separately testable development units with source-evidenced prerequisite edges. LOOPSBENCH comprises 112 tasks from authentic sources spanning 8 programming languages and 9 domains. Its flow-aware runtime releases tests along the ready frontier and retains completed nodes as regression obligations. We evaluate frontier coding agents paired with widely used loop implementations. The strongest configuration, Opus-4.7 with Claude Code and outer continuation, resolves 25.00% of tasks. Recorded plans recover only part of the source-recovered prerequisite DAG, and regression events remain visible across the evaluated loop profiles. We open source the benchmark data and code, including all tasks, more than 5,300 development units, and executable tests, at microsoft/Loopsbench.

Han Li, Zhemin Fang, Rili Feng et al. · 1 citation