人工 vs. 模型软标签学习与校准中的不确定性评估
An Assessment of Human vs. Model Uncertainty in Soft-Label Learning and Calibration
AI 导读
一篇 NeurIPS 2026 论文对 MNIST 及其合成变体上的软标签学习进行了受控审计，将人工软标签监督与标签模式偏移解耦。结果显示人工软标签确实带来准确率提升，但更大价值在于充当正则化器，改善困难样本上的模型校准并促进训练收敛稳定。数据集制图表明，人工软标签训练的模型能对齐人类不确定性，而合成标签训练的模型则不能。
正文 · 原文
Abstract:Central to human-aligned AI is understanding the benefits of human-elicited labels over synthetic alternatives. While human soft-labels improve calibration by capturing uncertainty, prior studies conflate these benefits with the implicit correction of mislabeled data (mode shifts), obscuring true effects of soft-labels. We present a controlled audit of soft-label learning across MNIST and a synthetic variant, re-annotating subsets to extract human uncertainty. By decoupling soft-label supervision from underlying label mode shifts, we show that while human soft-labels do provide accuracy gains, their larger value lies in acting as a regularizer that improves model calibration on difficult samples and promotes stable convergence across training runs. Dataset cartography reveals models trained on human soft-labels mirror human uncertainty, whereas those trained on synthetic labels fail to align with humans. Broadly, this work provides a diagnostic testbed for human-AI uncertainty alignment.
| Comments: | NeurIPS 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.18648 [cs.LG] |
| (or arXiv:2605.18648v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.18648 arXiv-issued DOI via DataCite |
Submission history
From: Maja Pavlovic [view email]
[v1]
Mon, 18 May 2026 16:55:21 UTC (27,207 KB)
[v2]
Mon, 5 Oct 2026 15:31:39 UTC (37,205 KB)
[v3]
Thu, 5 Nov 2026 08:34:40 UTC (37,205 KB)
来源：arXiv cs.AI · arxiv.org