Harbor Adapters are developed that port more than 80 benchmarks to evaluate arbitrary agents, and validated through rigorous code review and parity experiments, and Harbor-Index is introduced, a curated set of 82 difficult, diverse, and high-quality tasks spanning 29 benchmarks to support more reliable and comprehensiv...
Lin Shi, Hao-Wei Lin, Zi-Xuan Zhu et al.· 1 citation
This paper examines how skills may help address bottlenecks of current agents and how they may expand agent capabilities through reusable domain procedures loaded at inference time and outlines open questions in skill construction, composition, evaluation, portability, governance, and security.
Hanwen Xing, Haomin Zhuang, Xuandong Zhao et al.· Proceedings of the 32nd ACM...· 7 citations· ⚡1
This work proposes a self-contained two-stage training method, requiring no distillation from a stronger model, no manual annotation, and no pre-existing ICW IF ability, and introduces $\mathsf{ICWBench}$, a benchmark of three verifiable ICW instruction families, each scored on both detectability and answer quality.
Yepeng Liu, Tian-Yi Chen, Xuandong Zhao et al.· 0 citations
As Large Language Model (LLM) agents have demonstrated broad competence, but they still struggle in specialized, real-world workflows. Existing approaches such as RAG, fine-tuning and tool integration improve knowledge access, model adaptation, and external functionality, yet they do not fully address a central gap: th...
Hanwen Xing, Haomin Zhuang, Xuandong Zhao et al.· Proceedings of the 32nd ACM...· 6 citations· ⚡1
MatrAIx is introduced, a population-scale simulated-user evaluation infrastructure for testing AI systems and digital products with heterogeneous users and provides an end-to-end infrastructure for evaluating AI systems and digital products with diverse simulated human users.
Xiaomin Li, Yuexing Hao, Jian Hou et al.· 1 citation
This work introduces a novel approach, \textit{CanaryTrace}, to safeguard the ownership of text datasets and effectively detect unauthorized use by RA-LLMs, and demonstrates high query efficiency, detectability, and consistency, along with minimal perturbation to the original dataset, all without compromising the perfo...
Yepeng Liu, Xuandong Zhao, D. Song et al.· arXiv.org· 15 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
