LLM 能否恰当信任？评估对不完美伙伴的校准信任
Can LLMs Trust Appropriately? Evaluating Calibrated Reliance on Imperfect Partners
AI 导读
arXiv 论文提出一个包含 324 个场景、23 类协作类别的基准，研究大语言模型能否将伙伴证据转化为恰当的信任策略，每个场景含三次顺序决策及显式转移动态与成本安排。研究评估了 7 个模型和 36 个受控历史探针，发现最强前沿模型接近最优决策，其他模型则存在购买零期望信息价值的验证测试、过度采购保障、牺牲合作学习换取短期节省等病态行为。
正文 · 原文
Abstract:Large language models increasingly collaborate with humans and peer AI agents whose contributions may be incomplete, inconsistent, or incorrect. Effective collaboration requires calibrated trust, whereby decisions to rely on, verify, or assist a partner adapt rationally as empirical evidence accumulates. We investigate whether models translate partner evidence into appropriate reliance policies. We introduce a benchmark of 324 scenarios across 23 collaboration categories. Each scenario involves three sequential decisions with explicit transition dynamics, feedback emissions, and cost schedules, permitting exact analytical evaluation of choices against their expected consequences over the remaining interaction. Evaluating seven models alongside 36 controlled-history probes, we dissociate evidence sensitivity from normatively appropriate action. While the strongest frontier model approaches near-optimal decisions, other models exhibit recurring pathologies: procuring excessive safeguards, purchasing verification tests with zero expected value of information, prioritizing immediate savings over cooperative learning, and misallocating assistance to suboptimal channels. Supplying explicit posterior distributions resolves certain errors while introducing new failure modes. Calibrated trust in autonomous agents critically demands coupling empirical evidence with forward-looking decision utility.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2610.34107 [cs.AI] |
| (or arXiv:2610.34107v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34107 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jintao Huang [view email]
[v1]
Sun, 25 Oct 2026 07:06:31 UTC (474 KB)
来源：arXiv cs.AI · arxiv.org