认知正当性（epistemic warrant）：在无 ground truth 时评估 LLM 建议可依赖程度的框架
Epistemic Warrant for LLM Recommendations: Characterizing the Basis for Reliance When Ground Truth Is Unavailable
AI 导读
一项 arXiv cs.AI 研究提出“认知正当性（epistemic warrant）”这一决策层概念，用基于四层行为测试的“依赖证书（reliance certificate）”来刻画 LLM 建议的支持范围。验证显示证书赋值随预设正当性水平递增，更强的正当性与更高的独立人类共识相关，且其信息量超出模型口头表达的置信度。结论在不同模型和实现方式下保持稳健。
正文 · 原文
Abstract:Large language models are increasingly used to support organizational decisions. Despite their rapid and widespread adoption, users often lack a principled basis for assessing whether to rely on a specific recommendation. We introduce \emph{epistemic warrant}, a decision-level construct that characterizes the scope over which a recommendation is supported. We operationalize this construct through a reliance certificate that uses behavioral evidence from four tiers of tests to assess a recommendation's underlying warrant and assign it to a warrant category. We assess this operationalization using complementary forms of construct-validity evidence, including transformation validity, known-groups tests, independent human consensus, discriminant analyses, and robustness across models and alternative implementations. The results show that the certificate behaves in theoretically expected ways: its transformations instantiate their intended relationships, certificate assignments increase across groups prespecified to represent increasing levels of warrant, and stronger assigned warrant is associated with greater independent human consensus. Certificate assignments also capture information beyond verbalized confidence and are not readily explained by decision difficulty. The main conclusions remain robust across models and operationalizations. These findings support the reliance certificate as an operationalization of epistemic warrant for assessing the scope of support for a particular LLM recommendation when ground truth is unavailable.
| Comments: | 43 pages |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.04127 [cs.AI] |
| (or arXiv:2609.04127v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.04127 arXiv-issued DOI via DataCite |
Submission history
From: Shai Vardi [view email]
[v1]
Thu, 3 Sep 2026 17:25:20 UTC (253 KB)
[v2]
Thu, 29 Oct 2026 13:10:33 UTC (267 KB)
来源：arXiv cs.AI · arxiv.org