Large language model (LLM) agents are increasingly being used to automate machine learning research workflows, ranging from code development and experiment execution to research reproduction and scientific discovery, suggesting a shift from automating individual research tasks toward automating long-horizon workflows that transform research artifacts into executable ML systems.
Abstract
Large language model (LLM) agents are increasingly being used to automate machine learning (ML) research workflows, ranging from code development and experiment execution to research reproduction and scientific discovery. Recent systems demonstrate that LLMs can perform substantial portions of ML research engineering with limited human intervention. These advances suggest a shift from automating individual research tasks toward automating long-horizon workflows that transform research artifacts into executable ML systems. However, ML research engineering remains substantially more difficult than isolated code generation. A research artifact typically specifies a system only implicitly through a mixture of natural-language descriptions, equations, pseudo-code, implementation details, datasets, and evaluation protocols. An agent must first infer the intended computational structure, translate it into executable components, coordinate their dependencies, and repeatedly diagnose and repair failures during execution. Consequently, the quality of the final system depends not only on code generation, but also on whether the agent can maintain a consistent representation of what should be implemented throughout this long-horizon process.
Large language models (LLMs) equipped with terminal access have demonstrated strong capabilities in automating software engineering tasks. However, existing agents remain brittle on long-horizon workflows, where they must repeatedly reconstruct program state scattered across source files, configurations, tests, depende...
Hai-Bo Jin, Xin-Jie Li, Peng Kuang et al.· 0 citations
Context: Large language model (LLM) agents are increasingly used as software and data-engineering assistants, yet evidence about locally deployable open-weight agents remains limited. Existing evaluations often emphasize textual responses or isolated code generation rather than the validity of complete engineering arti...
Jorge García-Carrasco, Javier Sanchis, A. Reina-Reina et al.· 0 citations
Recent advances in large language models (LLMs) have enabled coding agents to autonomously perform complex software development tasks, yet their long-running execution remains difficult for users to inspect, understand, and steer. We present Vibetrace, an interactive trace visualization and intervention system for agen...
Xiao-Ke Yan, Jia-Ying Lei, Dan-Qing Shi et al.· 0 citations
E2E-SWE is introduced, a benchmark for evaluating whether coding agents can build complete, functional software repositories end to end and finds substantial variation in end-to-end repository generation ability, providing strong model differentiation while leaving considerable headroom for future progress.
A systematic audit of technical debt in AI-generated software is presented, revealing that AI does not eliminate flaws but rather introduces a distinct "machine signature" of defects.
Yue-Cai Zhu, Nikolaos Tsantalis, Peter C. Rigby· Proceedings of the 41st IEEE...· 0 citations
Enterprise customer-support agents increasingly rely on Large Language Models (LLMs) to execute bounded, tool-using workflows. We study this problem through the lens of Skill LLMs: specialized executors that operate within a selected skill context and must complete a narrow, stateful workflow under policy and tool cons...
Md Tahmid Rahman Laskar, Gundeep Singh, Xue-Yong Fu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.