arXiv cs.AI· Yijun Zhou, Minghao Shao, Ke Zhang, Renkai Yuan, Hongbo Yuan, Bin Hu, Bozhao Qian, Hongxia Shen, Yanhong Liu, Yuting Chen, Yuntao Shi, Zhongxuan Xu, Shuo Mao, Haijia Zhou, Zhendong Li, Tao Zhang, Chenggang Li, Yuxuan Wang, Zinan Zheng, Calvin Zhang, Jia Li, Heng Chang, Jie Feng·· 4 小时前AI 评分42
UnityBench-Mini3D：评估 LLM 智能体在 Unity 中的 3D 游戏开发能力
UnityBench-Mini3D: Evaluating LLM Agents for 3D Game Development in Unity
AI 导读
研究者推出 UnityBench-Mini3D 基准，包含 77 个项目级 Unity 任务，按 7 个专家定义技能类别和 3 个难度分级组织，用 1,178 个人工编写的里程碑测试做引擎内评估，无需主观 LLM 评审。
正文 · 原文
Authors:Yijun Zhou, Minghao Shao, Ke Zhang, Renkai Yuan, Hongbo Yuan, Bin Hu, Bozhao Qian, Hongxia Shen, Yanhong Liu, Yuting Chen, Yuntao Shi, Zhongxuan Xu, Shuo Mao, Haijia Zhou, Zhendong Li, Tao Zhang, Chenggang Li, Yuxuan Wang, Zinan Zheng, Calvin Zhang, Jia Li, Heng Chang, Jie Feng
Abstract:Game development stresses coding agents in ways general software engineering does not: correctness depends on how engine APIs, serialized scene state, and runtime behavior interact, not on source code alone. Existing benchmarks mainly target platforms such as Godot and heavily rely on LLM judges or coarse task-level outcomes. We present UnityBench-Mini3D, a benchmark of 77 project-level Unity tasks organized around seven expert-defined skill categories rather than tutorials or genres, with three difficulty tiers. Each task pairs a complete Unity project with natural-language requirements and is evaluated using 1,178 held-out, human-authored milestone tests covering scene structure, component configuration, and runtime behavior. This in-engine evaluation provides deterministic partial credit, localizes failures, and distinguishes partial implementations from complete solutions without relying on subjective judges. Across 21 models and 9 coding harnesses, the average pass rate reaches 80.9% on easy tasks but falls to 35.2% on hard tasks, with just 57.7% overall. UnityBench-Mini3D provides a reproducible, diagnostic testbed distinguishing systems across difficulty levels.
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.35980 [cs.SE] |
| (or arXiv:2610.35980v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35980 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Minghao Shao [view email]
[v1]
Mon, 26 Oct 2026 10:19:14 UTC (9,287 KB)
来源：arXiv cs.AI · arxiv.org