arXiv cs.AI· Zhongmou He, Yee Man Choi, Kexun Zhang, Jiabao Ji, Junting Zhou, Dejia Xu, Ivan Bercovich, Aidan Zhang, Yixiao Zeng, Lei Li·· 4 小时前AI 评分39
HARDTESTGEN:面向大语言模型算法代码的高质量 RL 验证器生成流水线
HARDTESTGEN: A High-Quality RL Verifier Generation Pipeline for LLM Algorithmic Coding
AI 导读
研究团队提出 HARDTESTGEN,为算法编程题自动合成高质量测试用例,用作强化学习的验证信号,并构建了含 26.6k 题目的数据集 HARDTESTS。相比现有测试,其验证 LLM 生成代码的精度提升 11.22 个百分点。用 HARDTESTS 验证器做拒绝采样和 RL 后训练可提升代码生成性能,数据集与流水线已开源。
正文 · 原文
Abstract:Verifiers provide important reward signals for reinforcement learning of large language models (LLMs). However, it is challenging to develop or create reliable verifiers, especially for code generation tasks. A well-disguised wrong solution program may only be detected by carefully human-written edge cases that are difficult to synthesize automatically. To address this issue, we propose HARDTESTGEN, an approach to synthesize high-quality test cases for algorithmic coding problems. We curate a comprehensive algorithmic programming dataset HARDTESTS with 26.6k problems and high-quality synthetic tests. Compared with existing tests, HARDTESTGEN tests demonstrate significantly higher accuracy in verifying LLM-generated code (+11.22 percentage points in precision, the percentage of actually correct code within the predicted correct ones). We also show that downstream post-training -- including rejection sampling and reinforcement learning (RL) -- using HARDTESTS verifier results in improved performance of LLM code generation. We open-source our dataset and synthesis pipeline at this https URL.
| Comments: | Original version was published at ICLR 2026. This is the expanded version with more details |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2505.24098 [cs.CL] |
| (or arXiv:2505.24098v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2505.24098 arXiv-issued DOI via DataCite |
|
| Journal reference: | He et al. HARDTESTGEN: A High-Quality RL Verifier Generation Pipeline for LLM Algorithimic Coding. International Conference on Learning Representations (ICLR), 2026 |
Submission history
From: Lei Li [view email]
[v1]
Fri, 30 May 2025 01:00:34 UTC (3,787 KB)
[v2]
Sun, 8 Nov 2026 18:53:49 UTC (6,173 KB)
来源:arXiv cs.AI · arxiv.org