PersonaScienceBench: Evaluating Scientific Agents with Diverse Simulated Researchers
TL;DR
PersonalityScienceBench evaluates how agents collaborate with diverse researchers alongside whether they complete the task, and across eighteen modelconfigurations, task performance, fact recovery, and self-repetition reveal different strengths.
Abstract
Scientific agents collaborate with researchers whose expertise and preferences differ, but benchmarks usually hold the collaborator fixed. We introduce PersonaScienceBench: 200 tasks from five benchmarks across four scientific domains. Simulated researchers provide context and feedback according to persona profiles, while task objectives and grading remain fixed. Agents encounter researcher preferences only through conversation. Across eighteen modelconfigurations, task performance, fact recovery, and self-repetition reveal different strengths. Detailed GPT 6 Astra analyses show distinct agent responses across researcher profiles; interaction traces illustrate targeted clar ification and workflow revision. These behaviors can differ even when final scores match. PersonaScienceBench therefore evaluates how agents collaborate with diverse researchers alongside whether they complete the task.
