跳到正文
arXiv cs.AI· Li Li, Xiangbo Gao, Bo Ni, Samyadeep Basu, Viet Dac Lai, Jing Shi, Seunghyun Yoon, Gang Wu, Yu Wang, Thien Huu Nguyen, Tyler Derr, Franck Dernoncourt, Zhengzhong Tu, Yue Zhao·· 2 小时前AI 评分39

面向指令引导视频编辑的运动感知奖励模型研究

Motion-Aware Reward Modeling for Instruction-Guided Video Editing

AI 导读

研究揭示指令引导视频编辑的奖励模型存在"运动坍塌"问题:在 VE-Bench 上,复合奖励在 97% 的案例中偏好抑制运动的候选结果。作者提出结合 RGB 帧差流与运动抑制、伪造运动负样本偏好训练的运动感知奖励模型,并用蒸馏锚点保持与人类质量判断一致。该模型在 VE-Bench 运动阶梯上把选择运动抑制候选的比率降至 0%,与人类质量评分的 Spearman 相关性达 0.57。

正文 · 原文

Authors:Li Li, Xiangbo Gao, Bo Ni, Samyadeep Basu, Viet Dac Lai, Jing Shi, Seunghyun Yoon, Gang Wu, Yu Wang, Thien Huu Nguyen, Tyler Derr, Franck Dernoncourt, Zhengzhong Tu, Yue Zhao

View PDF HTML (experimental)

Abstract:Reward models for instruction-guided video editing can favor temporal consistency at the expense of motion. We study this failure, termed \emph{motion collapse}, using controlled motion-suppression variants of real edits. On VE-Bench, the evaluated composite reward prefers a motion-suppressed candidate in $97\%$ of cases. Learned video rewards and a VLM judge also struggle with mild suppression, while latent optimization and reward-guided fine-tuning exploit this weakness to increase reward as motion declines. A linear probe of our base reward backbone performs near chance on mild suppression, motivating an explicit inter-frame dynamics input. We propose a motion-aware reward model that combines an RGB frame-difference stream with preference training against motion-suppressed and fabricated-motion negatives. A distillation anchor preserves alignment with human quality judgments. On the VE-Bench motion ladder, our model reduces the rate of selecting motion-suppressed candidates to $0\%$ while achieving a Spearman correlation of $0.57$ with human quality ratings. It also detects unseen suppression variants and preserves motion during reward-guided fine-tuning on two editing pipelines. These results support explicitly modeling motion in rewards for video editing.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.22495 [cs.CV]
  (or arXiv:2611.22495v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.22495

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Li Li [view email]
[v1] Tue, 17 Nov 2026 09:51:25 UTC (15,974 KB)

来源:arXiv cs.AI · arxiv.org