GDP.xlsx:用专业电子表格工作簿评测前沿大模型的新基准
GDP.xlsx: Benchmarking Frontier Models on Professional Workbooks
AI 导读
研究者发布 GDP_xlsx 基准,包含 13 个领域专业人士撰写的 70 项真实职场任务,配套工作簿共 218 万个单元格和 791 条二元评分标准,由可打开文件的智能体评审执行评分。
正文 · 原文
Abstract:We introduce GDP_xlsx, a benchmark of 70 realistic workplace requests written by professionals in 13 domains, each paired with an expert rubric of binary criteria and with the complete workbook or CSV file it refers to. Together the files hold 2.18 million populated cells, and the rubrics 791 criteria. Agents work in a container with spreadsheet tools, and an agentic judge that can open the file applies the rubric. Completeness and exclusivity criteria require set-valued answers to be found in full and with nothing extra. We rank models by strict pass@1, the share of attempts that meet every criterion. Across 20 model configurations, including frontier models from Google, Anthropic, and OpenAI, the top-scoring one, Gemini 4 Argon, passes only 38.3% of attempts, and the median score is 19.3%, leaving wide headroom. We release the evaluation harness and three public tasks; the other 67 are held out to protect the leaderboard from contamination.
| Comments: | 14 pages, 4 figures, 6 tables. Accepted to the Agent Evaluation Science Symposium (New York City, November 2026). Leaderboard: this https URL ; evaluation harness: this https URL ; public tasks: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.17766 [cs.CL] |
| (or arXiv:2611.17766v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.17766 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sushant Mehta [view email]
[v1]
Fri, 13 Nov 2026 05:24:50 UTC (23 KB)
来源:arXiv cs.AI · arxiv.org