Benchmarks Don't Talk Back:通过自我分歧学习辩证能力的研究
Benchmarks Don't Talk Back: Learning Through Self-Disagreement
AI 导读
arXiv 论文《Benchmarks Don't Talk Back: Learning Through Self-Disagreement》提出“辩证能力”概念:语言模型在错误反推下能否坚持正确立场并非由答题准确率决定,也无法靠推理时自反思或更多采样获得。
正文 · 原文
Abstract:A benchmark usually measures a language model alone with a question, and deployment rarely looks like that: answers get contested, and models give up correct ones under counter-reasoning that is itself wrong. We show that what fails there is not a shortfall of accuracy but a capability in its own right -- knowing when to hold a position and when to yield to a better one -- which we call dialectical competence. It stands on first-answer accuracy without following from it: a model can be made materially better at answering and no better at this, and training for accuracy alone can drift against it. Nor can it be bought at inference, by self-reflection or by sampling more. Taking a cue from the dialectical tradition in philosophy, in which productive opposition arises within a position rather than being imposed on it from outside, we hypothesize that this capability can be trained from opposition the model supplies itself. It states a position, generates the strongest case against that position -- its antithesis -- rules on whether the case stands, and is trained on that ruling, with no second model at training or at deployment. We call this learning through Self-Disagreement: reinforcement learning from verifiable rewards throughout, where the same check that scores an answer against its reference scores the ruling too, so no reward model, no preference data and no human annotation enter anywhere. The capability is trained rather than elicited: weak in the base model, it arrives with the reward, is not paid for out of first-answer accuracy, and holds against challengers the model never trained against, including ones stronger than itself. The opposition a model needs in order to improve, it turns out, need not come from outside it.
| Comments: | 34 pages, 5 figures, 15 tables |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.17563 [cs.CL] |
| (or arXiv:2611.17563v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.17563 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Soheil Abbasloo [view email]
[v1]
Thu, 12 Nov 2026 22:37:32 UTC (227 KB)
来源:arXiv cs.AI · arxiv.org