This work constructs DataPraxis-5K, a dataset of 5,000 verified trajectories spanning diverse domains and output formats, to train DataPraxis-27B and DataPraxis-35B-A3B, an analytics task synthesis framework based on verified workspace evolution.
Nan Jia, Wen-Xuan Shi, Jing-Qing Ruan et al.· 0 citations
Large language model (LLM) agents are increasingly used for long-horizon research and engineering, yet evaluating these capabilities remains expensive, time-consuming, and heavily dependent on human expertise. This motivates a natural question: can LLM agents themselves construct efficient benchmarks that preserve the...
Huai-Sheng Zhu, Hang-Fan Zhang, Teng Xiao et al.· 0 citations
This work proposes TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology that explicitly captures how tool invocations and environmental responses drive the divergence between effective strategies and failure modes.
It is found that starting performance and subsequent growth are associated with different capabilities: within a task category, similar early scores can precede different later gains, and a continuation policy to save time and compute with limited score loss is derived.
Hao-Yu Zheng, Zheng-Yu Chen, Huai-Sheng Zhu et al.· 0 citations
A time-truncation harness is proposed that enforces a temporal cut-off at every turn, enabling TIR-style sampling from historical events, enabling TIR-style sampling from historical events, reducing temporal leakage and reliance of rejection sampling or unsolved queries, increasing the sampling efficiency.
A systematic evaluation of seven frontier models on 36 long-horizon tasks based on a new framework that uses rule-based metrics to characterize within-run behavior through Solution Framing, Execution, and Feedback Control and controlled comparisons to assess experience reuse within and across tasks is presented.
Yi-Wei Li, Wanli Yang, He-Xiang Tan et al.· 1 citation
An extensive evaluation of automatic harness evolution for LLM agents is conducted, comparing harness evolution with simple test-time scaling and discovery baselines under comparable feedback and inference budgets, and also evaluating evolved harnesses on held-out tasks to assess whether the discovered improvements gen...
Yike Wang, Huaisheng Zhu, Zhengyu Hu et al.· arXiv.org· 18 citations
Student-Centric Answer Sampling (SCAS) is proposed, a framework that selects from verified teacher-generated answers according to their estimated student-centric learning cost and is derived by a token-wise gradient decomposition and used to guide answer selection during training.
Zhengyu Hu, Zheyuan Xiao, Linxin Song et al.· 0 citations
HarnessCompass is proposed, a novel automatic harness evolution framework built around constrained evolution, proactive feedback, and component-wise optimization that improves Pass@1 from 54\% to 66\% in only 5 evolution iterations, outperforming AHE in both effectiveness and evolution efficiency.
Luan Zhang, Ruo-Chen Zhou, Dan-Dan Song et al.· 10 citations· ⚡1
Co-Harness is introduced, a framework that jointly optimizes the agent harness and model parameters during post-training and suggests that joint harness and model optimization is an effective way to improve agents beyond fixed-harness post-training.
Zhengyu Chen, Teng Xiao, Huaisheng Zhu et al.· arXiv.org· 4 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.