arXiv cs.AI· Mingzhe Zheng, Yue Wu, Jing Tan, Chang Zou, Dengyang Jiang, Zeyue Tian, Xuanhua He, Ryan Xu, Yunyang Ge, Sihui Ji, Yunlong Lin, Kaihang Pan, Hongru Yan, Sixiang Chen, Dianyi Wang, Julian Zhang, Weijie Kong, Changlin Li, Lanqing Guo, Zhao Zhong, Qifeng Chen, Harry Yang·· 3 小时前AI 评分34
EmoReward：面向情感视频生成 RL 的自进化奖励校准机制
EmoReward: Self-Evolving Reward Calibration for Emotional Video Generation RL
AI 导读
研究者提出 EmoReward，一种自进化奖励校准训练机制，解决视频生成强化学习中情感表达难以评估、固定奖励易引发 reward hacking 的问题。该方法通过 Reflective Reward Gating 做实例级门控校准各信号贡献，并由 RewardMemory 从上一轮训练的奖励轨迹估计可靠性，指导下一轮多奖励权重分配。
正文 · 原文
Authors:Mingzhe Zheng, Yue Wu, Jing Tan, Chang Zou, Dengyang Jiang, Zeyue Tian, Xuanhua He, Ryan Xu, Yunyang Ge, Sihui Ji, Yunlong Lin, Kaihang Pan, Hongru Yan, Sixiang Chen, Dianyi Wang, Julian Zhang, Weijie Kong, Changlin Li, Lanqing Guo, Zhao Zhong, Qifeng Chen, Harry Yang
Abstract:Group Relative Policy Optimization (GRPO) has advanced generative model post-training in verifiable domains such as code and mathematics, but evaluating video quality, especially emotional expression, often depends on subjective aesthetics and has no uniquely correct target. Existing approaches typically rely on fixed rewards for visual quality, motion quality, and text alignment that poorly capture emotional intent and can invite reward hacking, while RLHF is difficult to scale and subject to individual aesthetic differences that complicate the calibration of facial, body motion, and chromatic cues across narrative contexts. We introduce EmoReward, a new self-evolving training mechanism that distills evidence reliability from reward traces collected during an actual training run to adaptively guide multi-reward weight allocation in subsequent training by determining which emotional evidence to trust and how much to weight it. EmoReward first plans specific grounding cues from the emotional prompt to guide evaluation, then uses heterogeneous signal primitives to score the generated video during the act stage. Reflective Reward Gating calibrates their contributions through instance-level gates for phase, face confidence, conflict, and subtlety, then renormalizes weights over the active set to produce the sample's fused reward. RewardMemory then estimates each primitive's reliability from rollout scores and outcomes within each emotional regime in the finished training run, then fixes bounded multipliers that calibrate fusion weights in the next training run without updating reward models. Experiments on HunyuanVideo 1.5 show improvements across multiple RL algorithms and compare EmoReward with other agentic reward methods under SAGE-GRPO. Ablations and transfer experiments on SD 3.5 and FLUX further support adaptive reward calibration for visual generation.
| Comments: | 30 pages, 15 figures |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.06518 [cs.CV] |
| (or arXiv:2611.06518v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.06518 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mingzhe Zheng [view email]
[v1]
Wed, 4 Nov 2026 13:43:09 UTC (28,772 KB)
来源：arXiv cs.AI · arxiv.org