RetroChessBench：用于重构与证明国际象棋历史的新基准
RetroChessBench: A Benchmark for Reconstructing and Proving Chess Histories
AI 导读
arXiv 论文提出 RetroChessBench，一个包含 12 项任务、面向回溯国际象棋的推理基准，涵盖精确长度构造、唯一终局与穷举答案集，并附带评分标准与终端奖励环境。
正文 · 原文
Abstract:Frontier language models such as GPT 6 Astra have contributed to research mathematics, demonstrating substantial reasoning capabilities. Yet serious reasoning failures persist even in tasks governed by explicit rules. We demonstrate this in retrograde chess which is a playground for pure reasoning, where solvers reconstruct legal histories and distinguish possible events from forced ones. We introduce RetroChessBench, a 12-task benchmark covering exact-length constructions, unique endings, and exhaustive answer sets, with references, rubrics, and a terminal-reward environment. In two trials per task and condition, GPT 6 Astra High fails to produce the required final answer in 22/24 attempts without tools and 18/24 with code execution; mean terminal RL rewards are 14.75% and 27.19%, respectively. The examples identify faulty deductions: failed responses reject legal histories and treat temporary obstructions as permanent. Online retrograde archives often provide puzzles and final move sequences without detailed derivations. We hypothesize that limited exposure to these missing explanations in web-derived training data contributes to the observed reasoning failures. This questions how far frontier mathematical reasoning really generalizes beyond familiar solution patterns. Claims of general intelligence require evidence that models can transfer learnt principles to areas with few worked examples.
| Comments: | 22 pages, 2 figures. Benchmark and RL environment: this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.39341 [cs.LG] |
| (or arXiv:2610.39341v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.39341 arXiv-issued DOI via DataCite |
Submission history
From: Tidor-Vlad Pricope [view email]
[v1]
Thu, 29 Oct 2026 04:32:11 UTC (51 KB)
来源：arXiv cs.AI · arxiv.org