Skip to content
Preprint

CLEAR: Structured Specifications for Controllable Long-Horizon ML Research Engineering

Nov 2026 · 0 citations · 26 references
Computer Science

TL;DR

Large language model (LLM) agents are increasingly being used to automate machine learning research workflows, ranging from code development and experiment execution to research reproduction and scientific discovery, suggesting a shift from automating individual research tasks toward automating long-horizon workflows that transform research artifacts into executable ML systems.

Abstract

Large language model (LLM) agents are increasingly being used to automate machine learning (ML) research workflows, ranging from code development and experiment execution to research reproduction and scientific discovery. Recent systems demonstrate that LLMs can perform substantial portions of ML research engineering with limited human intervention. These advances suggest a shift from automating individual research tasks toward automating long-horizon workflows that transform research artifacts into executable ML systems. However, ML research engineering remains substantially more difficult than isolated code generation. A research artifact typically specifies a system only implicitly through a mixture of natural-language descriptions, equations, pseudo-code, implementation details, datasets, and evaluation protocols. An agent must first infer the intended computational structure, translate it into executable components, coordinate their dependencies, and repeatedly diagnose and repair failures during execution. Consequently, the quality of the final system depends not only on code generation, but also on whether the agent can maintain a consistent representation of what should be implemented throughout this long-horizon process.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

Harness Engineering for Software Engineering via Modular Executable Dev-Primitives

Large language models (LLMs) equipped with terminal access have demonstrated strong capabilities in automating software engineering tasks. However, existing agents remain brittle on long-horizon workflows, where they must repeatedly reconstruct program state scattered across source files, configurations, tests, depende...

Hai-Bo Jin, Xin-Jie Li, Peng Kuang et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Evaluating Local Language Model Agents for Reproducible Data Engineering: An Empirical Software Engineering Study of Mobility Workflows

Context: Large language model (LLM) agents are increasingly used as software and data-engineering assistants, yet evidence about locally deployable open-weight agents remains limited. Existing evaluations often emphasize textual responses or isolated code generation rather than the validity of complete engineering arti...

Jorge García-Carrasco, Javier Sanchis, A. Reina-Reina et al. · 0 citations
#artificial intelligence Preprint Nov 2026

From Watching to Steering: Enabling Human-Coding Agent Collaboration through Execution Traces

Recent advances in large language models (LLMs) have enabled coding agents to autonomously perform complex software development tasks, yet their long-running execution remains difficult for users to inspect, understand, and steer. We present Vibetrace, an interactive trace visualization and intervention system for agen...

Xiao-Ke Yan, Jia-Ying Lei, Dan-Qing Shi et al. · 0 citations

E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch

E2E-SWE is introduced, a benchmark for evaluating whether coding agents can build complete, functional software repositories end to end and finds substantial variation in end-to-end repository generation ability, providing strong model differentiation while leaving considerable headroom for future progress.

Hantian Ding, Chloe Bi, Jia-Cheng Zhu et al. · 0 citations
Book Open access Oct 2026

Understanding the Code and Architectural Smells Generated by LLMs and Autonomous Agents

A systematic audit of technical debt in AI-generated software is presented, revealing that AI does not eliminate flaws but rather introduces a distinct "machine signature" of defects.

Yue-Cai Zhu, Nikolaos Tsantalis, Peter C. Rigby · 0 citations
#natural language process... Preprint Oct 2026

Evaluating LLMs as Agentic Workflow Executors in Real-World Customer Support Skills

Enterprise customer-support agents increasingly rely on Large Language Models (LLMs) to execute bounded, tool-using workflows. We study this problem through the lens of Skill LLMs: specialized executors that operate within a selected skill context and must complete a narrow, stateful workflow under policy and tool cons...

Md Tahmid Rahman Laskar, Gundeep Singh, Xue-Yong Fu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.