PersonaScienceBench：用多样化模拟科研人员评估科学智能体
PersonaScienceBench: Evaluating Scientific Agents with Diverse Simulated Researchers
研究者发布 PersonaScienceBench，包含来自五个基准、四个科学领域的 200 个任务，用带 persona 档案的模拟科研人员通过对话提供上下文和反馈，任务目标与评分保持固定。在 18 种模型配置上比较任务表现、事实恢复与自我重复，GPT 6 Astra 的分析显示智能体面对不同研究者画像会做出针对性澄清与流程调整，即使最终得分相同，这些行为也可能不同。
Authors:Siyang Wu, Weizhi Zhang, Chen Yang, Esther Xu Fei, Jorin Overwiening, Xu Pan, Jing Tang, Yixuan He, Kelvin C. Mo, Peiwen Li, Defu Cao, Xu Hu, Haixin Wang, Taoran Li, Haoming Li, Zheyuan Deng, Jingwen Zhang, Pengsong Zhang, Haichao Zhang, Guangze Luo, Yi Daniel Lu, Zihuai He, Chenxi Qiu, Dianzhuo Wang, Zhejian Peng, Shushu Liang, Wenyi Lu, Yifan Wang, Jintao Huang, Yijun Wang, Xiaomin Li, Yuexing Hao
Abstract:Scientific agents collaborate with researchers whose expertise and preferences differ, but benchmarks usually hold the collaborator fixed. We introduce PersonaScienceBench: 200 tasks from five benchmarks across four scientific domains. Simulated researchers provide context and feedback according to persona profiles, while task objectives and grading remain fixed. Agents encounter researcher preferences only through conversation. Across eighteen modelconfigurations, task performance, fact recovery, and self-repetition reveal different strengths. Detailed GPT 6 Astra analyses show distinct agent responses across researcher profiles; interaction traces illustrate targeted clar ification and workflow revision. These behaviors can differ even when final scores match. PersonaScienceBench therefore evaluates how agents collaborate with diverse researchers alongside whether they complete the task.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.32106 [cs.AI] |
| (or arXiv:2610.32106v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32106 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Siyang Wu [view email]
[v1]
Fri, 23 Oct 2026 03:11:01 UTC (237 KB)
来源：arXiv cs.AI · arxiv.org