arXiv cs.AI· Youpeng Wen, Gang Wang, Tianyu Huang, Weiwei Zhang, Zhijian Huang, Wei Chen, Xieyuanli Chen, Yunhui Liu·· 2 小时前AI 评分36
Orchestra 智能体框架:让机器人耦合动作执行与进度评估完成长程操作任务
Let the Mind Conceive, Let the Hand Achieve: Coupling Grounded Action and Progress Evaluation for Long-Horizon Manipulation
AI 导读
研究者提出 Orchestra 智能体框架,将任务规划与执行管理从动作生成中分离,可把多模态指令分解为有序子任务、追踪完成步骤并在执行中更新物体位置。
正文 · 原文
Abstract:A robot following a multi-step instruction needs to act on the right objects in the right order, even as the scene changes. It must also maintain awareness of task progress and determine when to advance. To address these challenges, we introduce Orchestra, an agent harness that separates task planning and execution management from action generation. Orchestra decomposes a complete multimodal instruction into ordered subtasks, tracks completed steps, and updates the locations of relevant objects and destinations during execution. It maintains a shared context containing the active subtask, spatial grounding, and recent observations. An action policy uses this context to generate actions, while a separately trained progress evaluator estimates when to advance to the next subtask. To evaluate whether robots trained on atomic instructions and short trajectories can execute longer subtask sequences, we introduce a benchmark named RMBench-Compose. Orchestra achieves a mean success rate of $68.22\%$ across nine RMBench tasks, outperforming Mem-0 by $26.22$ percentage points. On RMBench-Compose, it achieves an Average Success Length (ASL) of $7.35$ subtasks, $2.46$ times that of HarnessVLA. Across four real-world multi-step tasks, it improves mean ASL by $25.0\%$ over Orchestra using the base $\pi_{0.5}$ policy. These results show that Orchestra helps robots carry out longer sequences of subtasks while preserving the requested subtask order and acting on the specified objects.
Our code and project page are available at this https URL
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.18352 [cs.RO] |
| (or arXiv:2610.18352v2 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18352 arXiv-issued DOI via DataCite |
Submission history
From: Youpeng Wen [view email]
[v1]
Tue, 13 Oct 2026 13:20:59 UTC (16,845 KB)
[v2]
Tue, 17 Nov 2026 03:28:01 UTC (17,332 KB)
来源:arXiv cs.AI · arxiv.org