HIVE：基于在线验证的提示词选择方法，提升大语言模型推理 RL 训练效率

View PDF HTML (experimental)

Abstract:Reinforcement learning (RL) has become essential for post-training large language models (LLMs) in reasoning tasks. While scaling rollouts can stabilize training and enhance performance, the computational overhead is a critical issue. In algorithms like GRPO, multiple rollouts per prompt incur prohibitive costs, as a large portion of prompts provide negligible gradients and are thus of low utility. To address this problem, we investigate how to select high-utility prompts before the rollout phase. Our experimental analysis reveals that sample utility is non-uniform and evolving: the strongest learning signals concentrate at the ``learning edge", the intersection of intermediate difficulty and high uncertainty, which shifts as training proceeds. Motivated by this, we propose HIVE (History-Informed and online-VErified prompt selection), a dual-stage framework for data-efficient RL. HIVE utilizes historical reward trajectories for coarse selection and employs prompt entropy as a real-time proxy to prune instances with stale utility. By evaluating HIVE across multiple math reasoning benchmarks and models, we show that HIVE yields significant rollout efficiency without compromising performance.

Comments:	NeurIPS 2026 Camera Ready
Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2603.25184 [cs.LG]
	(or arXiv:2603.25184v4 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2603.25184 arXiv-issued DOI via DataCite

Submission history

From: Jiahao Wu [view email]
[v1] Thu, 26 Mar 2026 08:52:35 UTC (23,139 KB)
[v2] Sat, 6 Jun 2026 09:08:19 UTC (9,291 KB)
[v3] Wed, 2 Sep 2026 11:47:11 UTC (9,299 KB)
[v4] Wed, 28 Oct 2026 11:00:38 UTC (8,652 KB)