arXiv cs.AI· Zheng Zhang, Xiaoyang Tang, Shuaiqi Cheng, Jiakai Huang, Zhongbo Wu, Ziyi Zhang, Yuan Si, Zeyu Liu, Weibo Gao, Shunyu Liu, Shan You, Fei Wang, Xiaogang Wang, Dacheng Tao·· 3 小时前AI 评分39
WorldFailBench：世界模型能理解策略失败吗？
WorldFailBench: Do World Models Understand Policy Failures?
AI 导读
研究者发布 WorldFailBench 基准，检验世界模型能否忠实模拟机器人策略如何失败、在哪个阶段失败以及何时从成功转为失败。基准包含失败原因、失败阶段、失败边界三项评估任务，覆盖 LIBERO 和 RoboTwin 仿真环境以及 G1 人形机器人真实操作任务。对七个世界模型的实验显示其在理解策略失败方面存在明显差距。
正文 · 原文
Authors:Zheng Zhang, Xiaoyang Tang, Shuaiqi Cheng, Jiakai Huang, Zhongbo Wu, Ziyi Zhang, Yuan Si, Zeyu Liu, Weibo Gao, Shunyu Liu, Shan You, Fei Wang, Xiaogang Wang, Dacheng Tao
Abstract:Robot policy evaluation informs deployment decisions and policy improvement, but physical testing and simulator preparation require substantial effort. World models offer a complementary route through closed-loop interaction in generated environments. Recent benchmarks have demonstrated their potential as policy evaluators through outcome-based evaluation (i.e., assessing task success or failure). However, such evaluation does not establish whether world models faithfully simulate how policies fail. Misrepresenting failures can lead to incorrect diagnoses of policy weaknesses and make policies appear ready for deployment even when small scene changes cause them to fail. We therefore ask: do world models faithfully simulate how and at which stages policies fail, and under what conditions they transition from success to failure? To answer this question, we introduce WorldFailBench with three evaluation tasks. Failure reason tests whether models reproduce the observable errors policies make. Failure stage assesses whether failures concentrate at the same stages of full-task execution. Failure boundary examines whether models reproduce the conditions under which policies transition from success to failure under controlled scene changes in simulation. The benchmark covers manipulation tasks in the LIBERO and RoboTwin simulation environments and real-world manipulation tasks performed by the G1 humanoid robot. Experiments across seven world models reveal substantial gaps in their understanding of policy failures, providing guidance for developing more reliable world models for policy evaluation.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.33565 [cs.RO] |
| (or arXiv:2610.33565v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33565 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zheng Zhang [view email]
[v1]
Sat, 24 Oct 2026 09:33:52 UTC (6,632 KB)
来源：arXiv cs.AI · arxiv.org