arXiv cs.AI· Zhaoyu Chen, Hongnan Lin, Yongwei Nie, Fei Ma, Xuemiao Xu, Fei Yu, Chengjiang Long·· 5 小时前AI 评分34
Invert4TVG：以反转任务保持动作理解能力的时序视频定位框架
Invert4TVG: A Temporal Video Grounding Framework with Inversion Tasks Preserving Action Understanding Ability
AI 导读
Invert4TVG 是一种时序视频定位（TVG）框架，针对现有方法只优化时间 IoU、难以理解动作的问题，在强化学习框架中加入三类由 TVG 标注派生的反转辅助任务：动词补全、动作识别和视频描述。借助精心设计的奖励函数保持模型的动作理解能力，3B 模型在 Charades-STA 上 [email protected] 提升 7.1%，超过 SOTA 方法。
正文 · 原文
Abstract:Temporal Video Grounding (TVG) aims to localize video segments corresponding to a given textual query, which often describes human actions. However, we observe that current methods, usually optimizing for high temporal Intersection-over-Union (IoU), frequently struggle to accurately recognize or understand the underlying actions in both the video and query, thus reducing the effectiveness of these methods. To address this, we propose a novel TVG framework that integrates inversion-based TVG as auxiliary objectives to maintain the model's action understanding ability. We introduce three kinds of inversion TVG tasks derived from the original TVG annotations: (1) Verb Completion, predicting masked verbs (actions) in queries given video segments; (2) Action Recognition, identifying query-described actions; and (3) Video Description, generating descriptions containing query-relevant actions given video segments. These inversion tasks are entirely derived from the original TVG tasks and are probabilistically integrated with them within a reinforcement learning framework. By leveraging carefully designed reward functions, the model preserves its ability to understand actions, thereby improving the accuracy of temporal grounding. Experiments show our method outperforms state-of-the-art approaches, achieving a 7.1\% improvement in [email protected] on Charades-STA for a 3B model.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2508.07388 [cs.AI] |
| (or arXiv:2508.07388v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2508.07388 arXiv-issued DOI via DataCite |
Submission history
From: Zhaoyu Chen [view email]
[v1]
Sun, 10 Aug 2025 15:38:04 UTC (1,659 KB)
[v2]
Fri, 13 Feb 2026 06:37:58 UTC (7,015 KB)
[v3]
Thu, 29 Oct 2026 07:25:14 UTC (7,014 KB)
来源：arXiv cs.AI · arxiv.org