arXiv cs.AI· Kushagra Tiwary, David Mayo, Nikhil Behari, Xiangzhou Sun, Abdulrahman Alabdulkareem, Isaac Galatzer-Levy, Boris Katz, Brian Cheung·· 3 小时前AI 评分47
ArtifactArena：以实体世界的造物评估模型
ArtifactArena: Evaluating Models by What They Build in the Physical World
AI 导读
arXiv 论文提出 ArtifactArena，一个开放平台，让模型在物理仿真环境中进行软硬件协同设计挑战，工程制造可完全运行的机器人并在模拟竞技场中对抗。平台通过文本描述、物理仿真器反馈和对局数据三种 harness，评估前沿模型的零样本、验证器引导改进和开放式物理设计能力，并以机器人对抗赛的 Elo 排名作为基准。该框架与赛事基础设施将向社区持续开放提交，作为不饱和的开放式智能测试平台。
正文 · 原文
Abstract:To evaluate the frontier, we must measure models not by what they say, but by what they can engineer and build in grounded physical environments. We introduce ArtifactArena, an open-ended platform where models face a physically grounded hardware-software co-design challenge: engineering fully functional robots to compete in a simulated arena. We evaluate a frontier model's zero-shot, verifier guided refinement, and open-ended physical design capabilities through three harnesses that refine their bot artifacts based on text descriptions, physics simulator feedback, and gameplay data. We benchmark these capabilities with an Elo ranking of frontier models derived from head-to-head tournaments between their artifacts. By releasing this framework and tournament infrastructure for ongoing community submissions, we establish a living, non-saturating testbed to continuously measure the expanding limits of open-ended intelligence in the physical world. Please visit this https URL for more information.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.06511 [cs.RO] |
| (or arXiv:2610.06511v2 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.06511 arXiv-issued DOI via DataCite |
Submission history
From: Kushagra Tiwary [view email]
[v1]
Mon, 5 Oct 2026 15:28:48 UTC (7,125 KB)
[v2]
Thu, 22 Oct 2026 18:05:54 UTC (7,125 KB)
来源：arXiv cs.AI · arxiv.org