PersonaCodeBench：在角色驱动的人类交互会话中评测编码智能体
PersonaCodeBench: Evaluating Coding Agents in Persona-driven Human-Interactive Sessions
一篇 arXiv 论文提出 PersonaCodeBench，将用户角色特征转化为角色驱动的多轮编码会话中可验证的要求，用于评测编码智能体。该基准的角色池涵盖角色、技术能力、交互偏好与实际约束，由运行时控制器管理用户模拟器与要求披露，并用完成验证器和个性化验证器分别评估正确性与角色契合度，最终建立覆盖主要前沿智能体的排行榜。
Authors:Haixin Wang, Jianheng Hou, Hongyu Shen, Wenzhe Fan, Wei-Chieh Huang, Qianyu Zhu, Weichen Li, Taoran Li, Yuhang Yao, Ruohan Wang, Zhejian Peng, Wanting Jiang, Yifan Chen, Shushu Liang, Haoming Li, Defu Cao, Xu Hu, Siyang Wu, Chen Yang, Haodi Hu, Haichao Zhang, Cong Wang, Zhiyuan Yu, Weizhi Zhang, Yifan Wang, Jintao Huang, Yizhou Sun, Yijun Wang, Yuexing Hao, Xiaomin Li
Abstract:Coding agents must deliver correct software while respecting users' expertise, preferences, and working conditions. Existing benchmarks increasingly incorporate interaction, yet personalized requirement fulfillment remains underexplored. We introduce PersonaCodeBench, which turns user characteristics into verifiable obligations in persona-driven multi-turn coding sessions. Our persona pool spans roles, technical abilities, interaction preferences, and practical constraints, yielding task-grounded evolved requirements for implementation, permissions, testing, explanation, and delivery. At runtime, a controller governs user simulator responses and requirement disclosure, using online requirement evaluation to provide feedback and advance the interaction. Requirements emerge progressively, with their content, disclosure conditions, and acceptance criteria fixed before evaluation. A completion verifier and a personalization verifier separately assess native correctness and persona fulfillment. Joint pass requires both adherence to the declared budget and compliance with validity conditions. We evaluate all major frontier agents, establish a comprehensive leaderboard, and conduct detailed experimental analyses to validate effectiveness. PersonaCodeBench supports reproducible evaluation of coding agents' ability to meet explicit, personalized user expectations alongside coding objectives.
| Comments: | 40 pages, 9 figures |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.05739 [cs.AI] |
| (or arXiv:2611.05739v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05739 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Haixin Wang [view email]
[v1]
Tue, 3 Nov 2026 23:27:20 UTC (3,500 KB)
来源：arXiv cs.AI · arxiv.org