arXiv cs.AI· Hanwen Li, Jinhao Duan, Xiaoshuang Shi, Yue Zhang, Tianlong Chen, Kaidi Xu, Chenxi Yuan·· 3 小时前AI 评分41
UMAS：面向多智能体大语言模型系统的系统级不确定性量化框架
UMAS: System-Level Uncertainty Quantification for Multi-Agent LLM Systems
AI 导读
UMAS 是一个无参数、推理成本中性的系统级不确定性量化框架，通过每代生成的内在置信度估计智能体间影响的可信度，并将节点级不确定性聚合为系统级估计。在 Debate 和 DyLAN 设置的多项基准上，UMAS 平均比 SOTA 基线高出 10.54 AUROC 点，并支持幻觉检测与不确定性感知的答案选择，特定任务准确率最高提升 12.6 点。代码已开源，论文被 NeurIPS 2026 接收。
正文 · 原文
Abstract:Multi-Agent Systems (MAS) are a promising paradigm for improving the reasoning capabilities of large language models, yet they remain prone to hallucinations and erroneous outputs. Uncertainty Quantification (UQ) is crucial for assessing system reliability, but existing methods are largely designed for single-agent settings; how to exploit MAS structure for UQ while keeping inference cost-neutral remains underexplored. We propose UMAS, a parameter-free, cost-neutral, system-level UQ framework that estimates the credibility of inter-agent influence from each generation's intrinsic confidence. UMAS uses these credibility signals to modulate how uncertainty is updated at each node, and then aggregates node-level uncertainty into a system-level estimate. Extensive evaluations on Debate and DyLAN across diverse benchmarks show that UMAS consistently outperforms state-of-the-art baselines by an average of 10.54 AUROC points. Beyond uncertainty estimation, UMAS enables hallucination detection and uncertainty-aware answer selection, improving MAS accuracy by up to 12.6 points on specific tasks while enhancing reliability. Code is available at this https URL.
| Comments: | Accepted at NeurIPS 2026 |
| Subjects: | Multiagent Systems (cs.MA); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.33013 [cs.MA] |
| (or arXiv:2610.33013v1 [cs.MA] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33013 arXiv-issued DOI via DataCite |
Submission history
From: Hanwen Li [view email]
[v1]
Thu, 8 Oct 2026 04:05:08 UTC (7,029 KB)
来源：arXiv cs.AI · arxiv.org