arXiv cs.AI· Bo Yang, Zishuai Zhang, Weihua Li, Qinnan Zhang, Sijia Wen, Hainan Zhang·· 2 小时前AI 评分33
Guided-RLVR：带引导探索的可验证奖励强化学习方法
Guided-RLVR: Guided Exploration in Reinforcement Learning with Verifiable Rewards
AI 导读
研究人员提出 Guided-RLVR，在可验证奖励强化学习（RLVR）基础上增加一个轻量级控制器，向采样分布中注入语义上有意义的引导 token，促使模型从多个推理视角解决问题，从而扩展探索空间。针对 RLVR 推理轨迹坍缩到狭窄集合的问题，该方法在数学推理和代码生成基准上持续优于现有 RLVR 方法，并产生更多样化的推理轨迹。
正文 · 原文
Abstract:Reinforcement learning with verifiable rewards (RLVR) has shown strong empirical gains in improving the reasoning ability of large language models (LLMs) on tasks such as mathematics and programming. However, recent studies have shown that RLVR rollouts often collapse onto a narrow set of reasoning trajectories, limiting the explored reasoning space. Motivated by findings that thinking tokens at critical steps serve as critical structural elements in LLM reasoning pathways, we propose Guided-RLVR, a novel approach that augments RLVR with a lightweight controller. The controller strategically injects semantically meaningful steering tokens into the sampling distribution, encouraging the model to approach problems from multiple reasoning perspectives and thereby expanding the exploration space. Experiments on mathematical reasoning and code generation benchmarks show that Guided-RLVR consistently outperforms existing RLVR methods while producing more diverse reasoning trajectories.
| Comments: | 14 pages, 7 figures, 8 tables |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.37041 [cs.AI] |
| (or arXiv:2610.37041v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37041 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Bo Yang [view email]
[v1]
Tue, 27 Oct 2026 11:13:13 UTC (1,928 KB)
来源：arXiv cs.AI · arxiv.org