跳到正文
arXiv cs.AI· Seyed Mahmoud Sajjadi Mohammadabadi·· 4 小时前精选AI 评分63

arXiv 论文解析多智能体 LLM 中协调、集成与测试时计算的混淆

More Agents, More Samples, or More Thinking? Untangling Coordination, Ensembling, and Test-Time Compute in Multi-Agent LLMs

AI 导读

arXiv 论文综合九项实证评估共 516 个多智能体配置,发现此前评估常把协调收益与样本数量和推理计算混在一起。在按样本数或资源预算匹配基线时,多智能体系统准确率平均为 -0.042,74% 的可比对比中等于或低于基线;正收益需要分治证据、外部验证或跨模型误差多样性。论文提供审稿清单 pace CLI 与复现数据。

推荐理由

研究综合九项实证评估,指出多智能体对比基线不匹配时收益被高估,并提供审稿清单和核算工具,可帮助读者更严谨地评估多智能体系统。

正文 · 原文

View PDF HTML (experimental)

Abstract:Evaluations of multi-agent language models often compare communicating architectures against a single greedy pass. This practice confounds coordination with sample multiplicity and inference compute. We synthesize nine primary empirical evaluations comprising 516 unique multi-agent configurations organized along a resource vector R with six dimensions and a baseline ladder with four categories. Against baselines with a single pass (Category A), multi-agent accuracy differences are negative overall and near zero when omitting the largest contributing study. When evaluated against baselines matched on sample count (Category B) or resource budget (Category C), multi-agent systems average -0.042 in accuracy, and multilevel random effects modeling on the log odds scale remains negative across sensitivity checks. Multi-agent accuracy equals or falls below the baseline in 74% of these comparable contrasts, and study means are nonpositive in almost all evaluated studies. Across studies reporting both baselines, moving from a greedy pass to a sampled baseline shifts accuracy differences by -0.038 on average. Because communication adds no information when agents share identical evidence, positive returns require partitioned evidence, external verification, or error diversity across models. We provide a reviewer checklist and accounting utilities.
Comments: 43 pages, 11 figures, 13 tables. PACE Reviewer Checklist, evaluation utilities (pace CLI), and reproduction data at this https URL
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Multiagent Systems (cs.MA)
ACM classes: I.2.11; I.2.7
Cite as: arXiv:2611.17610 [cs.AI]
  (or arXiv:2611.17610v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.17610

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Seyed Mahmoud Sajjadi Mohammadabadi [view email]
[v1] Fri, 13 Nov 2026 00:18:25 UTC (478 KB)

来源:arXiv cs.AI · arxiv.org