AI agents are playing an increasingly important role in physics research. However, existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. We introduce FrontierPhysics, an end-to-end benchmark wit...
Bing-Ran You, Wen-Jun Ke, Ria Rosenauer et al.· 0 citations
Computer-use agents (CUAs) are commonly evaluated against fixed task specifications, but users may omit requirements, add constraints, or require approval before consequential actions. Completing the benchmark instruction does not guarantee serving the person who asked. We introduce PersonaCUABench, which holds 74 task...
Hao-Ming Li, De-Fu Cao, Xu-Ming Hu et al.· 2 citations
The evaluation of 20 frontier model arms demonstrates that high-fidelity user simulation is already attainable: leading models achieve up to 84.7% full-pass adherence under unprompted conditions.
Jin-Tao Huang, Yi-Fan Wang, Hong-Yuan Shen et al.· 1 citation
The ceiling of a task therefore answers when a transformer can succeed without its chain of thought, and the best accuracy that a shallow circuit can reach on a task the ceiling of the task, and a task is serial if its ceiling lies below one.
Jia-Shu He, Jin-Xuan Fan, Xiao Xiao et al.· 0 citations
This paper trains a recommender LLM to generate personalized explanations for its reccomendation, based on the user's watching history at a large video streaming service, and concludes that an LLM-based recommender can be fine-tuned on other complex tasks without compromising its original recommendation performance.
Jia-Shu He, Emma Kong, J. Tan et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.