This work systematically evaluates 16 PI configurations across three multi-turn agentic benchmarks and three student models that vary in scale and reasoning mode and finds that PI effectiveness depends on its content, generation source, and the student model.
Kai-Jian Zou, Emiliano Penaloza, MohammadHossein Rezaei et al.· 0 citations
Environments are the foundation of recursive self-improvement: they provide the problems agents work on and the feedback used to evaluate progress. Yet constructing challenging research environments with reliable evaluation still depends on domain experts, limiting their scale and disciplinary coverage. We introduce RS...
Ren-Xiong Wang, Darvin Yi, Abril Herrlein et al.· 2 citations
The increasing deployment of AI agents in long-horizon tasks yields massive execution logs. Diagnosing failures within these records is crucial for reliability, as it transforms outcome-level signals into actionable interventions. The sheer scale of the data renders human review impractical, driving the need for automa...
Harsh Raj, David Lee, Anas Mahmoud et al.· 0 citations
This work introduces an interaction-centric taxonomy that localizes failures to the interactions in which they originate and identifies the responsible component, and organizes 41 failure modes by assigning each to an edge between two components and a fault side indicating where the repair belongs.
Harsh Raj, Vipul Gupta, Anas Mahmoud et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.