We have 4 of 37 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
Repository benchmarks (e.g., SWE-bench) for coding agents often assume that lines of code edited can predict task difficulty, but such datasets'poor control over code and task types makes it hard to know which abilities truly drive agent errors. We present CABRA: a Coding Ability Blueprint for Rigorous Agent evaluation...
While agents aid initial task completion, they harm users' code comprehension and thus do not prepare users to extend their code, and low-effort agent interaction types, like copy+paste prompts and auto-accepted edits, are linked with lower comprehension.
VibeJam, a browser-based user study platform for users to collaborate with AI agents to develop websites, and open-source VibeJam to spur extensions and support studies on how coding agents can help users.
It is examined how alternatives to number right change what MCQA measures with six education-inspired schemes that assess abilities beyond accuracy: distractor elimination, abstention, confidence calibration, and self-correction.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
