多估计引出法并不能统一消除大语言模型判断偏差
Multi-Estimate Elicitation Does Not Uniformly Debias Language Model Judgments
AI 导读
一项 arXiv 研究比较了直接引出与单次回复内引出五个加权估计的多估计引出法，覆盖锚定、后见之明、基础比率、合取和现状/默认偏差任务，共 6 个配置、22,212 个正式测试单元。45 项预注册验证性检验中仅 12 项在完整样本上拒绝原假设，只有一项超过最小关注效应量：gpt-oss-120b 在多估计引出下结果后确认误差增加 10.68 个百分点。结论是现有证据不支持多估计引出能统一去偏差。
正文 · 原文
Abstract:Does eliciting five weighted estimates within a single response consistently improve numerical judgments? We compare direct and multi-estimate elicitation across anchoring, hindsight, base-rate, conjunction, and status-quo/default tasks in six frozen configurations. Each configuration contributes 618 admitted development units and 3,702 admitted formal units; the six-configuration formal cohort contains 22,212 units, including 21 missing outcomes. Of 45 preregistered confirmatory tests across five configurations, 12 reject on complete cases and 11 retain directional confirmation after the prespecified missingness-protection rule. Only one adjusted bound exceeds its smallest effect size of interest: gpt-oss-120b shows a 10.68-percentage-point increase in post-outcome confirmation error under multi-estimate elicitation (worst-case estimate 10.04 points). The remaining ten retained effects have sub-threshold point estimates; non-rejection does not establish equivalence. Joint presentation lowers signed conjunction excess in four confirmatory configurations, but this comparison changes both constraint visibility and call structure. Flash-high combines two disjoint execution-window batches under an approved continuation protocol, preserving their identities and non-causal interpretation. Post-hoc candidate-spread associations across all six configurations remain in-sample and require independent predictive validation. The completed cohort provides no evidence for uniform debiasing by multi-estimate elicitation. Materials: this https URL
| Comments: | 38 pages, 3 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.00597 [cs.CL] |
| (or arXiv:2611.00597v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.00597 arXiv-issued DOI via DataCite |
Submission history
From: An-Sheng Jhang [view email]
[v1]
Sat, 31 Oct 2026 03:46:00 UTC (2,777 KB)
[v2]
Thu, 5 Nov 2026 07:59:33 UTC (2,611 KB)
来源：arXiv cs.AI · arxiv.org