arXiv 论文提出近端目标方法:改进 HRM/TRM 循环精炼器在多正确答案问题上的训练
Proximal Targets: Training Looped Refiners on Problems with Many Correct Answers
AI 导读
arXiv 论文证明 HRM、TRM 等循环精炼器在题目存在多个正确答案时,深度监督的单一存储目标会设置无法通过增大模型容量消除的损失下限。作者提出近端目标(proximal target),让每一轮训练朝最接近自身输入的正确答案收敛。
正文 · 原文
Abstract:Looped refiners such as HRM and TRM apply one small network repeatedly to a draft answer, and deep supervision trains every pass toward one stored answer per puzzle. We prove that when a puzzle has many correct answers this target sets a loss floor that no model capacity removes, and that the loss-minimising pass blends the answers into an invalid grid. We propose the proximal target: train each pass toward the correct answer nearest to its own input, found by scanning a list of answers, by a search solver started at the draft, or within a band of quality set by a knob; correct answers become fixed points, and a pass only has to make fewer errors than its input has for the loop to converge, which its training loss bounds. With the same 1.7M-parameter network and 5 seeds, the proximal target solves 98.8% of Sudoku puzzles with 2-64 answers (standard target: 2.5%), 93.8% of sparse puzzles with more than 10,000 answers without any list of answers (0.0%), and 90.9% of 200-node 3-colourings after training on graphs of at most 50 nodes (36.9%), and different random starts end at different answers. Against GRAM, a stochastic-latent reasoner, a single run is correct far more often (98.8% vs. 57.7%) and twenty runs find more of the answers (53.6% vs. 37.4%), while GRAM is slightly ahead on its own 10-node colouring benchmark; in Scrabble the knob becomes a calibrated strength dial.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.22737 [cs.AI] |
| (or arXiv:2611.22737v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22737 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Syed Daimi Affan [view email]
[v1]
Tue, 17 Nov 2026 13:07:46 UTC (320 KB)
来源:arXiv cs.AI · arxiv.org