超越 0-100：置信度量表设计如何影响 LLM 的言语化置信与元认知
Beyond 0-100: How Confidence Scale Design Shapes Verbalized Confidence and LLM Metacognition
AI 导读
一项 arXiv 研究在 6 个 LLM 和 3 个数据集上考察言语化置信度量表设计，发现超过 78% 的回答集中在三个整数值上，量表选择并非中性。系统操纵粒度、边界位置和范围规律性后，0-20 量表在 meta-d' 指标上持续优于标准 0-100 格式，而边界压缩会降低表现，整数字偏好在不规则范围内依然存在。研究主张将置信度量表设计作为 LLM 评测的一等实验变量。
正文 · 原文
Abstract:Verbalized confidence, in which LLMs report a numerical certainty score, is widely used to estimate uncertainty in black-box settings, yet the confidence scale itself (typically 0--100) is rarely examined. We show that this design choice is not neutral. Across six LLMs and three datasets, verbalized confidence is heavily discretized, with more than 78\% of responses concentrating on just three round-number values. To investigate this phenomenon, we systematically manipulate confidence scales along three dimensions: granularity, boundary placement, and range regularity, and evaluate metacognitive sensitivity using $meta\text{-}d'$. We find that a 0--20 scale consistently improves metacognitive efficiency over the standard 0--100 format, while boundary compression degrades performance and round-number preferences persist even under irregular ranges. These results demonstrate that confidence scale design directly affects the quality of verbalized uncertainty and should be treated as a first-class experimental variable in LLM evaluation.
| Comments: | 20 pages |
| Subjects: | Artificial Intelligence (cs.AI) |
| MSC classes: | Natural language processing |
| Cite as: | arXiv:2603.09309 [cs.AI] |
| (or arXiv:2603.09309v4 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2603.09309 arXiv-issued DOI via DataCite |
Submission history
From: Yuyang Dai [view email]
[v1]
Tue, 10 Mar 2026 07:41:14 UTC (232 KB)
[v2]
Mon, 15 Jun 2026 11:58:00 UTC (222 KB)
[v3]
Thu, 10 Sep 2026 14:54:00 UTC (514 KB)
[v4]
Sat, 24 Oct 2026 15:33:55 UTC (514 KB)
来源：arXiv cs.AI · arxiv.org