arXiv cs.AI· Zijie Liu, Qi Yang Huang, Jinhao Duan, Yueh Che Lee, Thomas A. Joseph, Zayne Turner, Adam Seligman, Tianlong Chen, Zhaozhuo Xu·· 2 小时前AI 评分43
AgentOrch：全局编排与逐步交互的企业智能体差距研究
AgentOrch: How Close Can Global Orchestration Come to Step-by-Step Interaction?
AI 导读
AgentOrch 基准在 650 条企业 SOP 和 3,220 个运行时变体上，对比全局编排与逐步交互两种企业智能体执行方式。逐步交互对每个评测模型仍更准确，但从早期模型到最强近期模型，差距已缩小约 79%。三条路线可进一步缩小差距：增量式工作流构建平均提升 9.98 pp，带执行反馈修订完整工作流在留出变体上提升 6.13 pp，针对完整工作流的训练使五个模型平均提升 12.18 pp。
正文 · 原文
Abstract:Enterprise workflow agents typically rely on step-by-step interaction (Step-by-Step): the agent executes an action, observes its result, and invokes a large language model (LLM) again to decide the next step. Global orchestration (Global) instead generates a complete executable workflow in advance and executes it without further LLM calls, substantially reducing inference overhead, but it remains less accurate than Step-by-Step. We study how close Global can come to Step-by-Step and what narrows the remaining gap. We introduce AgentOrch, a benchmark that directly compares global orchestration with step-by-step interaction on 650 enterprise SOPs and 3,220 runtime variants under matched tools, environments, and evaluation. Step-by-Step remains more accurate for every evaluated model, but the gap has shrunk by about 79% from early models to the strongest recent model. We identify three routes to further narrowing this gap. Incremental workflow construction improves Global accuracy by an average of 9.98 percentage points (pp) without runtime execution feedback. Revising complete workflows with execution feedback improves all evaluated models, including 6.13 pp on held-out runtime variants. Targeted training on complete workflows improves accuracy by an average of 12.18 pp across all five trained models. Although a gap remains, global orchestration offers a promising path toward enterprise agents that deliver higher accuracy with fewer LLM calls.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.33375 [cs.AI] |
| (or arXiv:2610.33375v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33375 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zijie Liu [view email]
[v1]
Sat, 24 Oct 2026 00:08:11 UTC (1,532 KB)
来源：arXiv cs.AI · arxiv.org