arXiv cs.AI· Benjamin Pikus, Daniel Saunders, Mircea Mironenco, Salman Mohammadi, Mathew Mathew, Sirou Zhu, Divya Makkar, Xuhua Huang, Qinbo Li, Supreeth Narasimhaswamy, Siyuan Cen, Thomas Neff, Dejia Xu, Shyamal Buch·· 4 小时前AI 评分59
ZAP 用预训练小模型加仿真训练实现 96.7% 真机任务成功率
Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent
AI 导读
论文提出 Zero-real Agent Policy（ZAP），通过预训练 27B 视觉语言模型加工具调用框架，先监督微调再 GRPO 训练简单仿真任务，训练全程不使用任何真实机器人数据。
正文 · 原文
Authors:Benjamin Pikus, Daniel Saunders, Mircea Mironenco, Salman Mohammadi, Mathew Mathew, Sirou Zhu, Divya Makkar, Xuhua Huang, Qinbo Li, Supreeth Narasimhaswamy, Siyuan Cen, Thomas Neff, Dejia Xu, Shyamal Buch
Abstract:A robot policy needs to understand what a task is asking for in a scene, and how to move the robot to accomplish that task. To achieve these two skills, we introduce Zero-real Agent Policy (ZAP), a recipe that teaches a small off-the-shelf vision-language model to control a robot via a simple tool harness. We use a pretrained 27B parameter model to bootstrap world understanding and reasoning, followed by supervised fine-tuning and GRPO on simple simulated tasks to teach it how to move the robot. Without any real-world robot data during training, ZAP reaches 96.7% success on real-world tasks, matching GPT-6 Astra in our harness and outperforming the MolmoAct2 VLA (33.3%), while running 3x faster and 17x cheaper than GPT-6 Astra. We demonstrate that ZAP also generalizes to completely new objects, goals, strategies, and language by seating a test tube in a rack (and choosing a different hole in the rack when prompted), handing a test tube between arms mid-air, drawing an X on a tic-tac-toe board, and playing chess. These goals and types of manipulation don't appear in our training data. Furthermore, ZAP also transfers to an unseen embodiment. On a single-arm Franka, ZAP reaches 55.9% on RoboLab-120, 4.9 percentage points ahead of the leading VLA/WAM baseline.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.35817 [cs.RO] |
| (or arXiv:2610.35817v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35817 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Benjamin Pikus [view email]
[v1]
Mon, 26 Oct 2026 17:44:34 UTC (1,914 KB)
来源：arXiv cs.AI · arxiv.org