Software development with coding agents often begins with a rough request that users refine after trying the implementation. Final correctness alone does not capture the repeated guidance this process requires. We introduce FeatureLoop, a benchmark of 61 human-reviewed tasks from 34 repositories across five primary pro...
Jia Feng, Alyssa Wu, Cui-Yun Gao et al.· 0 citations
A software feature usually evolves over time: developers extend its first version as new use cases and requirements arrive, and each extension builds on code written earlier. Most feature benchmarks evaluate a single request on a fixed codebase. Feature evolution requires evaluating successive extensions on an agent's...
Yun-Long Wang, Alyssa Wu, Cui-Yun Gao et al.· 0 citations
This initial study demonstrates that directly prompting off-the-shelf code LLMs remains ineffective; even advanced prompting strategies cannot compensate for the lack of task-specific knowledge, and fine-tuning proves highly effective, with pseudo-code representation consistently yielding the best performance.
Qingyuan Li, Bin-Chang Li, Cuiyun Gao et al.· ACM Transactions on Software...· 3 citations· ⚡1
A Translation fRAmework Via rule-guided reasoning and rEinforcement Learning, namely TRAVEL, consisting of two modules that employs Monte Carlo Tree Search-based reasoning path construction guided by Rust-specific rules, and introduces reinforcement learning that couples execution feedback with reasoning-quality signal...
