Coding agents must deliver correct software while respecting users'expertise, preferences, and working conditions. Existing benchmarks increasingly incorporate interaction, yet personalized requirement fulfillment remains underexplored. We introduce PersonaCodeBench, which turns user characteristics into verifiable obl...
Hai-Xin Wang, Jian Hou, Hong-Yuan Shen et al.· 0 citations
PersonaChatBench is introduced, a framework for evaluating task competence and responsiveness to diverse user needs that provides a basis for evaluating consistency across users and guiding the development of assistants that better accommodate diverse needs in everyday applications.
Zhe-Yuan Deng, Tao-Ran Li, Jing-Wen Zhang et al.· 0 citations
PersonalityScienceBench evaluates how agents collaborate with diverse researchers alongside whether they complete the task, and across eighteen modelconfigurations, task performance, fact recovery, and self-repetition reveal different strengths.
Si-Yang Wu, Wei-Zhi Zhang, Chen Yang et al.· 0 citations
Computer-use agents (CUAs) are commonly evaluated against fixed task specifications, but users may omit requirements, add constraints, or require approval before consequential actions. Completing the benchmark instruction does not guarantee serving the person who asked. We introduce PersonaCUABench, which holds 74 task...
Hao-Ming Li, De-Fu Cao, Xu-Ming Hu et al.· 2 citations
The evaluation of 20 frontier model arms demonstrates that high-fidelity user simulation is already attainable: leading models achieve up to 84.7% full-pass adherence under unprompted conditions.
Jin-Tao Huang, Yi-Fan Wang, Hong-Yuan Shen et al.· 1 citation
Results show that post-training can substantially improve financial forecasting performance, together with changes in how the model investigates the market, on this outcome-selected benchmark.
Jia-Cheng Guo, Suo-Zhi Huang, Shu-Zhen Li et al.· 0 citations
ContextEvo, a framework that learns a context policy from long-horizon trajectories and reconstructs the model-visible context at key decision points, identifies context-related failures, and applies targeted policy updates, is introduced.
AQuA, which comprises two separate language-model-driven research systems: one for symbolic factor discovery and one for trainable model development, is presented, which comprises two separate language-model-driven research systems that implement recursive self-improvement at the level of the research process.