证明步骤验证何时值得付出代价？面向开源权重数学推理的认证或拒绝验证运行条件分析
When Is Proof-Step Verification Worth Its Cost? Operating Requirements of Certify-or-Decline Verification for Open-Weight Mathematical Reasoning
AI 导读
一篇 arXiv 论文评估了基于开源权重模型的证明步骤验证流程：将解题答案改写为类型化步骤后，由 LLM 裁判认证或拒绝。在 GPQA 和 AIME-2025 上，认证使答案精确率分别提升 31 和 46 个百分点，但仅覆盖 35% 和 21% 的尝试；在接近饱和的 MATH-500 上最多提升 5 个点且损失 13-27 个点覆盖率。
正文 · 原文
Abstract:Proof-step verification rewrites a solver's answer into typed steps and uses LLM judges to certify or decline. We evaluate this pipeline on open-weight models across GPQA, AIME-2025, and a MATH-500 subset. Certification raises answer-key-match precision by 31 percentage points on the strong GPQA configuration and 46 on AIME, while answering only 35% and 21% of attempts. On near-ceiling MATH-500, gains are at most 5 points at a loss of 13-27 points of coverage. An output-token-matched self-consistency selector obtains 82.9% precision at 35% coverage, compared with the verifier's pooled 87.6% (paired difference: +4.8 points, 95% interval [-8.2, 18.5]). A diagnostic sample reveals a distinct failure mode: 52 of 88 completed problems end in invalid proof construction without ever reaching a judge. Finally, external-LLM acceptance of rendered certified chains is 61.0-67.6%, compared with 87.6% answer-key match. These audits lack human or formal-kernel ground truth. Open-weight auditor pairs show near-zero chance-corrected agreement, which measures consistency rather than accuracy against proof truth. These results motivate separate reporting of execution failures, selected-answer precision, and independently assessed proof validity. We provide per-configuration results and evaluation details in the supplement.
| Comments: | 13 pages, 1 figure, 13 tables. Accepted at the 6th Workshop on Mathematical Reasoning and AI (MATH-AI) at NeurIPS 2026. Code: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.14794 [cs.AI] |
| (or arXiv:2611.14794v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14794 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yashica Patodia [view email]
[v1]
Tue, 10 Nov 2026 22:29:57 UTC (43 KB)
来源：arXiv cs.AI · arxiv.org