PerfAgent is presented, a profiler-guided, verifier-in-the-loop workflow that gives an off-the-shelf coding agent the feedback needed to find real hotspots, improve beyond the first passing patch, and use profiler evidence rather than timing alone to decide what to optimize next.
Abstract
Large language model (LLM) agents now perform well on correctness-oriented repository-level tasks, including SWE-Bench issue resolution and feature implementation in real codebases. However, they still struggle with repository-level code optimization, which requires preserving behavior while improving runtime performance. Passing tests is not enough in this setting; a patch must preserve behavior, implement code optimization, and approach expert speedups. Current agents often miss bottlenecks hidden behind abstraction layers and native extensions, stop after shallow speedups, or insufficiently test the code patches that thus may silently break edge cases. We present PerfAgent, a profiler-guided, verifier-in-the-loop workflow that gives an off-the-shelf coding agent the feedback needed to find real hotspots, improve beyond the first passing patch, and use profiler evidence rather than timing alone to decide what to optimize next. On two challenging optimization benchmarks, GSO and SWE-fficiency-Lite, PerfAgent more than doubles the rate of expert-matching patches over OpenHands with GPT-5.1, improving from 19.6% to 39.2% on GSO and from 26% to 74% on SWE-fficiency-Lite. It also surpasses an oracle best-of-five baseline at substantially lower cost, showing that the gains come from better feedback rather than additional test-time sampling.
The results indicate that functional correctness in code generation can be meaningfully improved without modifying the backbone architecture, by jointly optimizing how tasks are prompted, how the model is adapted, and how final outputs are selected.
This paper identifies patch verbosity as a major yet overlooked concern in LLM-based APR and proposes RECAP, a lightweight, plug-and-play adapter that attaches to existing repair frameworks after generation that achieves a substantially better size-correctness tradeoff.
Wenqiang Luo, J. Keung, Xiaoyu Shi et al.· 0 citations
SCOPE is presented, a prover-initialized subgoal critic for code generation that adapts a Lean-oriented prover model to produce three parseable feedback fields for downstream code generation: subgoals, gap analysis, and a robustness checklist.
Yueke Zhang, Yifan Zhang, Zihan Fang et al.· 0 citations
RUCACoder is proposed, a closed-loop multi-agent framework with a Retriever for hierarchical repository exploration, a Verifier for reranking and validation, and a Coder for feedback-driven script synthesis that consistently outperforms strong retrieval and generation baselines.
Kaitao Lin, Songwen Gong, Adam Jatowt et al.· Annual International ACM SIG...· 1 citation
SWE-RPG is introduced, a repository-level benchmark that combines executable patch evaluation with validated ground-truth references (GTs) for Requirement Clarification and Implementation Planning, and suggests implicit-requirement recovery as a key candidate direction for improving coding agents.
Natural-language requirements for program synthesis are often incomplete or ambiguous, yet large language models are commonly expected to generate code in a single pass. Prior clarification-based methods address this issue by asking follow-up questions when sampled candidate programs disagree, but fixed clarify-on-disagreement policies can overuse clarification and can also overtrust weak behavioral agreement. We present an adaptive routing framework for LLM-based program synthesis that treats clarification as an inference-time control decision. The framework augments a ClarifyGPT-style pipeline with execution-driven confidence estimation, semanticdifference analysis, and bounded candidate expansion, allowing the system to choose among direct generation, additional evidence gathering, and clarification. We evaluate the framework on MBPP, HumanEval, and extended-test variants using GPT-4.1 mini, Claude Haiku 4.5, and GPT-5.4 mini. Adaptive routing improves pass@1 accuracy by up to 7.60 percentage points over single-pass baselines. Compared with fixed-policy clarification, it preserves accuracy while reducing token usage by up to 57.2% for GPT-4.1 mini, and reallocates computation toward harder cases for Claude Haiku 4.5. These results suggest that clarification is most useful when triggered selectively based on execution evidence and semantic disagreement, even when ambiguity is observed indirectly through candidate behavior rather than through explicitly annotated ambiguous requirements.
Muhammad Ahmed, Edwar Tiu, Niyati Nikunj Kapadia et al.· Annual International Compute...· 0 citations