Coding agents must deliver correct software while respecting users'expertise, preferences, and working conditions. Existing benchmarks increasingly incorporate interaction, yet personalized requirement fulfillment remains underexplored. We introduce PersonaCodeBench, which turns user characteristics into verifiable obl...
Hai-Xin Wang, Jian Hou, Hong-Yuan Shen et al.· 0 citations
PersonaChatBench is introduced, a framework for evaluating task competence and responsiveness to diverse user needs that provides a basis for evaluating consistency across users and guiding the development of assistants that better accommodate diverse needs in everyday applications.
Zhe-Yuan Deng, Tao-Ran Li, Jing-Wen Zhang et al.· 0 citations
Computer-use agents (CUAs) are commonly evaluated against fixed task specifications, but users may omit requirements, add constraints, or require approval before consequential actions. Completing the benchmark instruction does not guarantee serving the person who asked. We introduce PersonaCUABench, which holds 74 task...
Hao-Ming Li, De-Fu Cao, Xu-Ming Hu et al.· 2 citations
The evaluation of 20 frontier model arms demonstrates that high-fidelity user simulation is already attainable: leading models achieve up to 84.7% full-pass adherence under unprompted conditions.
Jin-Tao Huang, Yi-Fan Wang, Hong-Yuan Shen et al.· 1 citation
COEVO is introduced, a framework that updates model parameters from on-policy experience while adapting contextual guidance according to the state of the evolving policy, and suggests that external context should be viewed not merely as a fixed interface to a large language model, but as an adaptive component of recurs...
Si-Wei Chen, Xin-Ping Bao, Xin-Yu Cai et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.