arXiv cs.AI· Julian Poeltl, Thomson Yen, Patrick Shea, Shubham Anil Varma, Dhruv Rastogi, Zihao Chen, Mengshi Zhang, Hongseok Namkoong·· 2 小时前AI 评分50
MBABench-2.0：评测 AI 智能体构建超出简单可验证性的金融电子表格
MBABench-2.0: Evaluating AI Agents on Constructing Financial Spreadsheets Beyond Simple Verifiability
AI 导读
MBABench-2.0 基准包含 101 个原创金融电子表格构建任务，最难任务估计需 3 年以上经验建模者耗时 33 小时。该基准以 12 大类 129 项标准评估准确性之外的可读性、可维护性和可审计性质量。对 8 个模型、4 种 harness 的 20 多个智能体评测显示，最优智能体通过率仅 16%，暴露出当前智能体电子表格构建能力的显著差距。
正文 · 原文
Abstract:Spreadsheets are the lingua franca of modern finance, driving valuation, forecast, audit, and many other core workflows of the industry. As LLM agents advance to automate knowledge work, spreadsheet construction is a natural and pertinent testbed. However, most existing spreadsheet benchmarks evaluate agents on fill-in-the-blank style tasks, where an agent performs minor edits in otherwise complete spreadsheets. This protocol fails to capture the most central and time-consuming part of the spreadsheet workflow: the modeling decisions, format design, and sheet layout regular knowledge workers need to toil through to build a workbook from scratch. We introduce MBABench-2.0, a benchmark of 101 original spreadsheet construction tasks, the hardest of which is estimated to take 33 hours for a modeler with 3+ years of experience. Unlike coding or math, where a unit test or a final answer can verify an agent's output, the quality of a financial spreadsheet that concerns readability, maintainability, or auditability, is not so simply verifiable. To holistically evaluate spreadsheet construction, we introduce 129 criteria spanning 12 categories that operationalize quality beyond the accuracy of the spreadsheet. Evaluating over 20 agents across 8 models and 4 harness types, we find that frontier agents rarely construct spreadsheets that are both accurate and meet our quality criteria: the top agent achieves a pass rate of only 16% on the benchmark. The benchmark demonstrates a substantial gap in spreadsheet building capability of current agents.
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.31711 [cs.SE] |
| (or arXiv:2610.31711v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2610.31711 arXiv-issued DOI via DataCite |
Submission history
From: Thomson Yen [view email]
[v1]
Thu, 22 Oct 2026 02:30:05 UTC (2,385 KB)
来源：arXiv cs.AI · arxiv.org