LLM 评判模型把队友错误报告误判为幻觉：对多智能体 LLM 系统的断言感知重新评分研究

View PDF HTML (experimental)

Abstract:LLM judges that score hallucination flag any statement that contradicts a reference fact. Agents in multi-agent LLM systems quote and dispute each other's claims, so such a judge can count a correct report of a teammate's error as a hallucination. An earlier version of this study, a preprint (arXiv:2606.21666v1), reported that broadcasting raised a hallucination rate from 0.492 to 0.658 (+34%) and called it contamination; this version withdraws that claim, and the preprint's runs cannot be re-scored. An unpublished revision of that harness reported 0.289 to 0.554 (+0.265, 95% CI [0.225, 0.305]), and its saved outputs are re-scored here. Under an analysis plan fixed before any judge call, three judges from three vendors labeled each of six facts in 1,832 outputs as asserted wrong, attributed to another source, asserted correct or absent. On values that outputs assert as their own, the broadcast effect was -0.0208 (95% CI [-0.0458, +0.0041]) on 10 Full-Broadcast trials: not distinguishable from zero, and equivalent to zero only within a post hoc margin of +/-0.0663 and under the plan's prompt. On these LLM-judge labels, reports of a teammate's wrong value accounted for nearly all of the Haiku travel effect. Scored for mentions, the judges reproduced the effect (+0.1996). With Llama agents, 2 of 40 teammate outputs asserted the wrong destination. One of twelve Claude judge-prompt combinations gave an asserted increase (Sonnet 4.6, one paraphrase: +0.0246, 95% CI [+0.0055, +0.0436]). In 40 new trials per arm, judged by Claude models like the agents, the asserted difference was -0.0505 when both judges agreed and +0.0161 (95% CI [+0.0024, +0.0299]) when either did. Main limit: the labels have no human validation; human labels for 120 outputs are pending. The outputs come from one harness and 13 hand-written scenarios, so the shares do not transfer.

Comments:	40 pages, 2 figures. Replacement of v1 (11 pages, 1 figure), posted after a re-score of a later revision under an analysis plan fixed before the judge calls found no support for v1's headline contamination claim; v1's own runs cannot be re-scored; see the correction note under the abstract
Subjects:	Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multiagent Systems (cs.MA)
ACM classes:	I.2.11
Cite as:	arXiv:2606.21666 [cs.AI]
	(or arXiv:2606.21666v2 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2606.21666 arXiv-issued DOI via DataCite

Submission history

From: Carson Rodrigues [view email]
[v1] Fri, 19 Jun 2026 18:17:45 UTC (46 KB)
[v2] Thu, 12 Nov 2026 06:37:36 UTC (73 KB)