判定不变而承诺漂移:识别推理模型的有害 Overthinking
Conserved Verdicts, Drifting Commitments: Identifying Harmful Overthinking
AI 导读
一项 arXiv 研究提出用“承诺-判定差”(commitment-verdict gap)识别推理模型的有害 overthinking:仅靠 final-correctness probes 估计的判定无法决定何时停止。
正文 · 原文
Abstract:A reasoning model that has reached a correct answer can lose it by continuing to think. Deciding whether to stop means comparing two probabilities at the same point of a chain: that the answer the model would give if stopped now is correct (its commitment), and that the answer it will finish with is correct (its verdict). We show that the verdict alone, which is what final-correctness probes estimate, cannot make this comparison. As a forecast of a fixed outcome it has no predictable drift, whether continuing helps or hurts. The commitment-verdict gap equals the expected remaining change in answer quality and the accuracy gained by stopping. A second identity splits any accuracy-versus-budget curve into answer change within chains and change in which chains are still running. We measure both on MATH-500. At all 379 points where DeepSeek-R1-Distill-Qwen-1.5B holds a correct answer, selected without regard to how its chain ends, continuing loses 3.6 points on average (95% CI [2.2, 5.2]), and the loss concentrates in a few problems. In six reasoning models from three families, on MATH-500 and OlympiadBench and with budgets up to 16,384 tokens, declining accuracy curves come from which chains remain, while the chains still reasoning keep or improve their answers. An oracle stopping rule gains at most +1.8 and +2.5 points at 1.5B and 7B. The framework says what to measure before building a stopping rule.
| Comments: | 40 pages |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.21946 [cs.AI] |
| (or arXiv:2611.21946v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21946 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jianghong Zhou [view email]
[v1]
Mon, 16 Nov 2026 22:08:54 UTC (633 KB)
来源:arXiv cs.AI · arxiv.org