跳到正文
arXiv cs.AI· Zhongmou He, Yee Man Choi, Kexun Zhang, Jiabao Ji, Junting Zhou, Dejia Xu, Ivan Bercovich, Aidan Zhang, Yixiao Zeng, Lei Li·· 4 小时前AI 评分39

HARDTESTGEN:面向大语言模型算法代码的高质量 RL 验证器生成流水线

HARDTESTGEN: A High-Quality RL Verifier Generation Pipeline for LLM Algorithmic Coding

AI 导读

研究团队提出 HARDTESTGEN,为算法编程题自动合成高质量测试用例,用作强化学习的验证信号,并构建了含 26.6k 题目的数据集 HARDTESTS。相比现有测试,其验证 LLM 生成代码的精度提升 11.22 个百分点。用 HARDTESTS 验证器做拒绝采样和 RL 后训练可提升代码生成性能,数据集与流水线已开源。

正文 · 原文

View PDF HTML (experimental)

Abstract:Verifiers provide important reward signals for reinforcement learning of large language models (LLMs). However, it is challenging to develop or create reliable verifiers, especially for code generation tasks. A well-disguised wrong solution program may only be detected by carefully human-written edge cases that are difficult to synthesize automatically. To address this issue, we propose HARDTESTGEN, an approach to synthesize high-quality test cases for algorithmic coding problems. We curate a comprehensive algorithmic programming dataset HARDTESTS with 26.6k problems and high-quality synthetic tests. Compared with existing tests, HARDTESTGEN tests demonstrate significantly higher accuracy in verifying LLM-generated code (+11.22 percentage points in precision, the percentage of actually correct code within the predicted correct ones). We also show that downstream post-training -- including rejection sampling and reinforcement learning (RL) -- using HARDTESTS verifier results in improved performance of LLM code generation. We open-source our dataset and synthesis pipeline at this https URL.
Comments: Original version was published at ICLR 2026. This is the expanded version with more details
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:2505.24098 [cs.CL]
  (or arXiv:2505.24098v2 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2505.24098

arXiv-issued DOI via DataCite

Journal reference: He et al. HARDTESTGEN: A High-Quality RL Verifier Generation Pipeline for LLM Algorithimic Coding. International Conference on Learning Representations (ICLR), 2026

Submission history

From: Lei Li [view email]
[v1] Fri, 30 May 2025 01:00:34 UTC (3,787 KB)
[v2] Sun, 8 Nov 2026 18:53:49 UTC (6,173 KB)

来源:arXiv cs.AI · arxiv.org