QREF, a plug-and-play Query Refiner and Evidence Filter that rewrites search queries according to the current reasoning intent while ensuring complementary coverage across each search batch, improves the performance of DeepSearch agents across four challenging benchmarks.
Yu-Jian Chen, Yu-Hui Wang, Jun-Zhe Wang et al.· 0 citations
LiveMACEBench is introduced, a process-aware benchmark that uses live financial markets as a naturally evolving testbed for persistent LLM agents, demonstrating that mechanism access, effective mechanism use, and downstream performance are not interchangeable measures of agent capability.
Jun Zhao, Lei-Ming Fu, Yan Wen et al.· 0 citations
EvoIn is an agent fine-tuning framework that bridges evolution and internalization, and consistently enables agents to learn stronger decision-making procedures, raising the pass rate by 10.9 points in-domain and by 9.2 points out-of-domain.
Shi-Han Dou, Shao-Hua Liu, Zhong-Hang Lu et al.· 0 citations
SciHorizon-eLab is introduced, an agentic protocol-to-task compiler that formulates scientific embodied task construction as a compilation problem and constructs \BenchName, a ready-to-use benchmark comprising 300 certified tasks across diverse laboratory operations.
Mao-Kai Qin, Chuan Qin, Qi Zhang et al.· 0 citations
ExplorationBench turns the wicked problem of evaluating scientific exploration into a concrete and tractable framework built on verifiable Alien Worlds, and finds that the strongest systems can acquire and apply unfamiliar rules, while performance varies substantially across trajectories and continued exploration can s...
Ming Zhang, Zhen Xiang, Pei-Zhong Gao et al.· 0 citations
Results show that persistent strategy experience provides effective guidance for automated fine-tuning on unseen tasks, and propose Strategy Accumulation and Guided Execution (SAGE), a two-stage framework that makes automated fine-tuning search cumulative.
Haoran Zhao, Wei Du, Dingwen Yang et al.· 0 citations
ReDraft (Reference-Driven Revision and Fine-Tuning) shows that revising the model's own rollout rather than directly imitating an expert trajectory can reconcile cold-start acquisition with prior-capability retention.
Zhi-Hao Zhang, Ming-Qi Wu, Qiaole Dong et al.· 0 citations
SPIEval is introduced, a human-curated benchmark grounded in five cognitive capabilities that exposes fundamental limitations of current LLM-based mobile assistants and motivates future research in this direction.
Junjie Ye, Zhuohui Sheng, Shao-Hua Liu et al.· 0 citations
SWE-Bench Pro Verified offers a more trustworthy benchmark for assessing software engineering agents, which combines anti-hacking safeguards that eliminate major leakage channels without disrupting normal agent functionality, with task refinement that minimally corrects inconsistencies within flawed instances.
Pujun Zheng, Zi-Xin Shang, Shufan Jiang et al.· 1 citation
Beacon, a novel agentic visual reasoning model trained with supervised fine-tuning and reinforcement learning, is proposed, which achieves the highest average score among the evaluated open-source models and ranks first on 11 benchmarks.
Qi-Xun Wang, Yang Shi, Le-Tian Cheng et al.· arXiv.org· 1 citation
A novel textual representation of fault trees is proposed, and a benchmark for multi-turn dialogue systems that emphasizes robust interaction in complex environments is constructed, evaluating a model's ability to assist in malfunction localization.
Yuhui Wang, Zhi-Xiong Yang, Ming Zhang et al.· arXiv.org· 0 citations
AgentCompass is introduced, an open-source, lightweight, and extensible infrastructure for evaluating LLM-based agents that organizes the evaluation process around three independent components, thereby enabling flexible configurations without requiring the reimplementation of complex execution logic.