arXiv cs.AI· Yizhu Chen, Jianguang Jiang, Jiameng Huang, Chuyu Fang, Bojun Zhang, Zhenyu Jiao, Xing Yuan, Tong Lin·· 2 小时前AI 评分29
RACE：策略与评分标准的奖励自适应协同演化框架
Reward-Adaptive Co-Evolution of Policy and Rubric
AI 导读
arXiv 论文提出 RACE，一种用于开放式强化学习的奖励自适应协同演化框架。该框架在组层面将 rollout 组分为信息性、多样性坍缩或评分标准失败三类并据此路由训练信号，在阶段层面基于奖励饱和触发评分标准优化，并采用冻结 judge 和尺度归一化的成对边际奖励。在统一评测协议下，RACE 取得最高平均性能，在全部八项任务上超过最强固定交替基线，同时提升评分标准的判别力。
正文 · 原文
Abstract:Rubric-based rewards provide fine-grained, interpretable supervision for open-ended reinforcement learning, where holistic scalar rewards struggle to capture multidimensional response quality. Static rubrics can lose discriminative power as the policy distribution evolves, while existing policy--rubric co-evolution methods rely on globally prescribed update schedules that ignore heterogeneous reward informativeness. We introduce RACE, a Reward-Adaptive Co-Evolution framework. At the group level, per-rollout-group training-signal routing classifies groups as informative, diversity-collapsed, or rubric-failed, updating the policy only with informative groups while caching rubric failures. At the phase level, event-triggered phase switching enters rubric optimization upon persistent reward saturation and returns when rubric-reward improvement plateaus. Rubric phases train the generator on cached pairs using a frozen judge and a scale-normalized pairwise-margin reward. Across both levels, online threshold adaptation adjusts routing and switching thresholds through proportional control. Under a unified evaluation protocol, RACE achieves the highest average performance and outperforms the strongest fixed-alternation baseline on all eight tasks, while improving rubric discrimination.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.40096 [cs.AI] |
| (or arXiv:2610.40096v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.40096 arXiv-issued DOI via DataCite |
Submission history
From: Yizhu Chen [view email]
[v1]
Thu, 29 Oct 2026 14:00:07 UTC (363 KB)
来源：arXiv cs.AI · arxiv.org