在无形式化答案领域训练“边界保持”能力：证据充分性判断中奖励训练的对照 knock-in 研究
Trainable Boundary-Keeping Where No Formal Answer Exists: A Controlled Knock-In Study of Reward Training in Evidence-Sufficiency Judgment
AI 导读
一项预注册的同基座 14B 对照 knock-in 研究测试了无正确性奖励领域的证据充分性判断，比较 SFT+GRPO 的边界保持方案、乱序配对安慰剂与通用推理模型。
正文 · 原文
Abstract:Reasoning-tuned and reward-trained models are often assumed to verify more reliably because they reason longer or receive correctness signals. We test this in evidence-sufficiency judgment: deciding whether available information actually supports a causal claim, a domain with no decidable correctness reward. Using a pre-registered, same-base, controlled knock-in design, we compare a boundary-keeping recipe (SFT then GRPO) with its own baseline, a scrambled-pairing placebo, and a general-purpose reasoning model under an action-pressure protocol with no evaluation invitation. All load-bearing outcomes are rated by two separate model-rating sessions masked to arm labels; automatic judges only trigger. The trained arm raises spontaneous correct refusal from 25% to 100% (40 per arm; Wilson 95% CI lower bound 91.2%, non-overlapping with the baseline interval [14.2%, 40.2%]), while the scrambled placebo collapses (5% spontaneous / 7.5% invited correct; 91.1% false refusal). The effect is boundary-specific rather than blanket refusal: automatic false refusal on 281 well-formed claims is 1.1% (Wilson upper bound 3.09%), and a targeted 18-item arm-(c) recheck (3 flagged plus 15 accepted) yields 0/18 and 1/18 false refusals in two sessions. The dissociation is the main result: final boundary-keeping is trainable without an inference-time verifier, while observed active numeric/calculation checking in the H3 assay remains near zero; SFT saturation makes the GRPO increment not separately identifiable. A pre-registered mechanism adjudication favors reward internalization of final-action status (H2, best-supported, not proven). The scope is a controlled 14B same-family study: we do not claim proof, cross-family generality, or independent human-expert validation; the rating sessions are model sessions, not human reviewers.
| Comments: | 14 pages, 3 figures. Aggregate data/figure code: this https URL (code MIT; data/text/figures CC BY 4.0). Writing/implementation assisted by AI agents; the author designed the study and is responsible for the final text |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.35226 [cs.AI] |
| (or arXiv:2610.35226v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35226 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: YaJie Yin [view email]
[v1]
Mon, 26 Oct 2026 11:16:28 UTC (98 KB)
来源：arXiv cs.AI · arxiv.org