arXiv cs.AI· Yifei Yao, Yude Zou, Guanbo Jiao, Xing Gao, Ning Gao, Yukai Wang, Jiantong Chen, Zihou Zhu, Xinyu Li, Ningyu Yan, Shuai Wang, Jianming Ma, Haoyu Guo, Weinan Zhang, Chunhua Shen·· 4 小时前AI 评分41
用 EBench 评测前沿智能体 GPT-6-Astra：从直接控制到策略调度
Profiling a Frontier Agent with EBench: From Direct Control to Harness with Policy
AI 导读
研究者在机器人基准 EBench 上评测 GPT-6-Astra 驱动的机器人控制智能体，比较直接控制与调用预训练 VLA/WAM 策略的 harness 两种配置。直接控制下整体表现与预训练策略相当，短时程任务领先、泛化波动最小，但长时程、桌面操作和高精度接触任务落后；harness 模式结合两者优势，将整体任务成功率提升至最强预训练策略水平并显著降低 token 成本，但仍低于各任务单独上限。
正文 · 原文
Authors:Yifei Yao, Yude Zou, Guanbo Jiao, Xing Gao, Ning Gao, Yukai Wang, Jiantong Chen, Zihou Zhu, Xinyu Li, Ningyu Yan, Shuai Wang, Jianming Ma, Haoyu Guo, Weinan Zhang, Chunhua Shen
Abstract:Frontier multimodal large language models (MLLMs) can control robots without robot-specific training, but how their capabilities compare and interact with post-trained vision-language-action (VLA) and world action model (WAM) policies across manipulation demands remains to be thoroughly evaluated. In this report, we evaluate GPT-6-Astra on EBench as the core of an agent that controls a robot through an agent harness under the same visual and proprioceptive observation protocol as post-trained policies. We study two agent configurations: direct control, in which the agent generates waypoint commands directly, and harness with policy, in which it can also invoke a post-trained VLA or WAM policy for action execution. Under direct control, the agent achieves competitive overall performance relative to post-trained policies. It leads on short-horizon tasks and exhibits the least performance variation across generalization dimensions, whereas it underperforms post-trained policies on long-horizon tasks, tabletop manipulation, and high-precision contact-rich tasks. In the harness with policy, invoking a post-trained VLA or WAM combines the complementary strengths of both components, raising overall task success to the level of the strongest compared post-trained policies while substantially reducing token cost. However, performance remains below the per-task upper bound of the standalone components due to suboptimal policy invocation, fixed-instruction execution without subgoals, and grasp disruption when the agent resumes control. Overall, our study characterizes the capability boundary of a frontier MLLM agent and suggests directions for improving future agent harness design. Project page: this https URL
| Comments: | Project page: this https URL |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.35832 [cs.RO] |
| (or arXiv:2610.35832v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35832 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yifei Yao [view email]
[v1]
Mon, 26 Oct 2026 17:51:02 UTC (11,767 KB)
来源：arXiv cs.AI · arxiv.org