E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch
Coding agents powered by large language models (LLMs) are evolving from making localized code changes to developing complete software repositories. However, evaluating repository-scale generation remains challenging: tasks must demand system-level reasoning while ensuring that all evaluated behaviors are precisely spec...
Hantian Ding, Chloe Bi, Jia-Cheng Zhu et al.
· 0 citations