arXiv cs.AI· Qizhi Chu, Zekai Yu, Sijie Wen, Yang Liu, Chen Qian, Cheng Yang, Chuan Shi, Zhiyuan Liu·· 4 小时前AI 评分35
MASBench：在部分可观测条件下评测基于 LLM 的多智能体协作
MASBench: Benchmarking LLM-based Multi-Agent Collaboration under Partial Observability
AI 导读
研究者推出 MASBench，一个在部分可观测约束下评测 LLM 多智能体协作的基准，覆盖 Reasoning、Scheduling 和 Game 三类渐进式任务。它评估 Protocol、Memory、Routing 三种协作机制，并提供性能分数、通信成本和成本效益等确定性指标。跨多个 LLM 骨干模型的实验为有效的多智能体系统设计提供实证指导，代码已开源。
正文 · 原文
Abstract:Large language models (LLMs) have progressively evolved into the core of autonomous agents. Building on this progress, LLM-based multi-agent systems (MAS) coordinate multiple agents into a synergistic team to accomplish complex tasks that exceed the capabilities of individual agents. The effectiveness of such systems depends not only on the agents themselves, but also on how collaboration mechanisms are designed and organized. Note that real-world collaboration is typically partially observable, where each agent can only access partial information about the environment due to physical or privacy-related constraints. However, many existing multi-agent benchmarks assume global observability, and leave limited support for systematically evaluating collaboration mechanisms. To bridge this gap, we introduce MASBench, a multi-agent collaboration benchmark designed under partially observable constraints. It is organized into three progressive task categories: Reasoning, Scheduling, and Game. Through this structure, we progressively evaluate three representative collaboration mechanisms: Protocol, Memory, and Routing. MASBench further provides deterministic evaluation metrics, including performance score, communication cost, and cost effectiveness, to characterize both collaboration outcomes and communication overhead. Experiments across diverse LLM backbones and mechanism configurations offer empirical guidance for effective MAS design. Code is available at: this https URL
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.04672 [cs.AI] |
| (or arXiv:2610.04672v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.04672 arXiv-issued DOI via DataCite |
Submission history
From: Qizhi Chu [view email]
[v1]
Sat, 3 Oct 2026 17:40:24 UTC (780 KB)
[v2]
Wed, 4 Nov 2026 01:46:04 UTC (780 KB)
来源：arXiv cs.AI · arxiv.org