叠更多关卡：如何让 AI 像人一样通用玩《超级马里奥》
Stack more levels: how to get general and human-like Mario playing
AI 导读
一项 arXiv 研究训练 PPO 智能体在规模递增的 PCG 生成《超级马里奥》关卡上训练，并在 1000 个留出测试关卡上评估泛化能力。研究发现，增加训练关卡数量通常能提升未见关卡的表现；再用 DRAIL 结合人类游戏示范微调后，智能体的动作层面更接近人类，同时保留了 runner、killer、collector 三种 PPO 预训练形成的玩法风格。
正文 · 原文
Abstract:Super Mario Bros. is widely used in game AI and procedural content generation (PCG), yet it remains unclear whether a model-free agent can reliably win unseen levels. To address this question, we train Proximal Policy Optimization (PPO) agents on increasingly large sets of PCG-generated Mario levels and evaluate their generalization on 1000 held-out test levels. We then shape competent PPO policies into three gameplay archetypes---runner, killer, and collector. Finally, we fine-tune these PPO policies using Diffusion-Reward Adversarial Imitation Learning (DRAIL) with human gameplay demonstrations. We find that increasing the number of distinct PCG-generated training levels generally improves unseen-level performance, and that fine-tuning PPO policies with DRAIL increases action-level human-likeness while preserving much of the persona induced by PPO pretraining.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.16175 [cs.AI] |
| (or arXiv:2611.16175v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.16175 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuchen Li [view email]
[v1]
Thu, 12 Nov 2026 02:06:30 UTC (102 KB)
来源：arXiv cs.AI · arxiv.org